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FOREWORD 


This  report  was  prepared  by  the  American  Institute  for  Research  under  USAF 
Contract  No.  AF  33 ( 6l6 ) - 2080 * The  contract  was  initiated  under  Project 
7197*  "Human  Engineering  Factors  in  the  Design  of  Training  Equipment."  The 
contract  was  administered  by  the  Psychology  Branch,  Aero  Medical  Laboratory, 
Directorate  of  Research,  Wrigit  Air  Development  Center,  with  Dr.  Gordon  A. 
Eckstrand,  Chief,  Trainer  Research  Section,  Psychology  Branch,  acting  as 
Project  Scientist. 

It  is  intended  for  use  by  engineers  who  design  training  equipment  or  assist 
in  preparing  the  specifications  for  them,  by  psychologists ' and  others  who  are 
concerned  with  the  design  of  programs  of  training  which  use  training  devices 
and  aids.  It  is  possible  that  soma  instructors  will  be ’interested  in  at  least 
some  of  the  content,  since  the  instructor  is  perhaps  the  most  ubiquitous  of 
all  training  devices  available  and  may  compensate  for  defects  and  exploit  the 
assets  of  hardware  used  for  training. 

The  purpose  of  this  report  is  to  assist  those  who  specify,  design,  or  use  train- 
ing devices  to  consider  the  human,  psychological  factors  which  determine  the 
effectiveness  of  the  devices.  A number  of  rule  of  thumb  recommendations  are 
offered,  usually  in  a condensed  conoext  of  explanation.  For  a more  extended 
treatment  of  most  topics  in  this  report,  see  the  parent  reports  of  which  this 
is  a digest.  These  parent  reports  are  listed  in  the  Preface. 

The  author  is  indebted  to  Dr.  Eckstrand  for  suggestions  on  form  of  presentation 
and  for  careful  editorial  review  and  to  Dr.  Alan  D.  Swain  especially  for  the 
material  on  the  instructor's  station.  Dr.  Swain  is  author  of  the  parent  report 
of  which  this  instructor's  station  material  is,  in  great  part,  a digest. 
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ABSTRACT 


A training  device  is  a machine  whose  purpose  is  to  teach  job  skills  which  will 
transfer  to  operational  situations.  As  such,  the  human  factors  involved  in 
efficient  learning  and  transfer  of  training  are  considerations  essential  to 
'•-oonorny  and  training  value  of  trainer  design.  The  report  presents  a number  of 
considerations  and  recommendations  for  trainer  design  under  the  following  topics? 
X.  Some  principle  concepts  in  learning  and  transfer  of  learning!  II.  Froblems 
of  physical  simulation;  III.  Stage  of  learning  and  degree  of  physical  simu- 
.lotion;  IV,  Knowledge  of  results  and  scoring;  V,  Recording  procedures; 

VI,  Proficiency  measurement;  VII,  The  design  of  the  instructor's  station; 

VIII,  The  trainer  as  demonstrator  of  principles;  IX.  Outline  of  steps  in  de- 
signing a training  device. 

This  report  is  essentially  a digest  of  previously  published  material  by  the 
author  and  by  Dr.  Alan  D,  Swain,  It  has  been  adapted  for  persons  who  may  not 
have  a professional  background  in  psychology.  The  report  is  intended  for  those 
who  participate  in  framing  the  requirements  for  training  devices,  designing 
them  and,  in  lesser  degree,  for  those  who  use  them  in  trainingo. 


PUBLICATION  REVIEW. 

This  report  has  been  reviewed  and  is  approved. 
FOR  THE  COMMANDER: 


ato -Mhc. /**<«, M 


/ JACK  BOLLEhlfD 
/^Colonel,  USAF  (MC) 


Chief,  Aero  Medical  Laboratory 
Directorate  of  Research 
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PREFACE 


’Several  publications,  currently  available  or  anticipated  in  the 
near  future,  indicate  the  features  of  equipment  ana  work  environments 
that  help  or  hinder  a human  operator  to  oo  a job.  This  report  is 
intended  to  suppliment  those  publications  nnu  emphasizes  those  human 
properties  and  their  matching  environmental  requirements  which  deal 
with  his  learning  to  do  a job. 

Theoretically,  what  a human  can  do  ana  what  he  can  learn  to  do 
are  part  of  a single  problem.  A task  which  is  easy  to  uo  often  means 
that  it  is  easy  to  learn.  For  example,  a task  which  presents  immediate 
knov/ledge  of  results  tends  (other  things  being  equal)  to  be  more  readily 
learned  as  well  as  easier  to  perform  reliably  at  a high  accuracy  lev^l 
Practically,  the  uifferentiation  is  made  between  phases  of  activity 
called  "training11  and  phases  of  activity  called  "operational  performance." 

In  order  to  design  equipment  efficient  for  training  purposes  it  is 
therefore  necessary  to  be  familiar  with  those  processes  by  which  human 
beings  learn,  and  with  those  conditions  generally  favorable  to  learning. 
Unfortunately  for  the  designer's  art,  those  properties  and  conditions 
cannot  now  be  stated  in  quantitative  form.  Design  must  therefore  be 
guided  more  by  principles  about  the  learning  process  than  quantitative 
prescription. 

For  this  reason  the  text  in  1 his  report  must  be  somewhat  more  dis- 
cursive than  handbooks  and.  guides  that  deal  with  "operational  performance", 
a subject  matter  in  hunr.n  behavior  less  complex  and  more  readily  control- 
led than  "training."  That  xs,  the  behavior  possibilities  in  learning  to 
operate  are  clearly  more  numerous  than  those  in  operating  the  device. 
Because  of  this  lack  of  nard-boiled  Prescriptions  in  training,  it  is  often 
necessary  to  solve  problems  through  awareness  of  variables  and  processes. 
The  first  chapter  of  this  report  represents  a minimum  of  terms  and  ideas 
for  maximum  interpretation  of  the  braining  process. 

This  first  chapter,  like  the  rest  of  the  report,  is  a digest  and 
summary  of  more  thorough  treatments  of  training  and  the  aesign  of  train- 
ing devices.  The  principal  reference  Documents  are  the  following: 

Miller,  Robert  B.  A Method  for  Determining  Human  Engineering  Design 
Requirements  for  Training  Equipment.  WADC  Technical  Report  53- 
135 . Wright  Air  Development  Center,  Air  Research  "md  Development 
Command,  US/iF,  Wright-Patterson  Air  Force  Bass  uuo,  1953* 

Handbook  on  Training  and  Training  Equipment  Design,  WADC  Techni- 
cal Report  53-136.  Wright.  Air  Development  Center,  Air  Research 
and  Development  Command,  USAF,  Wright- Pat berson  Air  Force  Base, 
Ohio,  1953. 

A Method  for  Man-Machine  Task  Analysis.  WADC  Technical  Report 
53-137.  Wright  Air  Development  Center  Air  Research  and  Develop- 
ment Command,  USAF,  Wright-Patterson  Air  Force  Base,  Ohio,  19 53 • 
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Human  Engineering  Design  Schedule  for  Training  Equipment.  WAJDC 
Technical  Report  53-138.  Wright  Air  Development  Center,  Air 
Research  and  Development  Command,  USAF,  Wright-Patterson  Air 
Force  Base,  Ohio,  1953* 

Swain,  Alan  D.  Guide  for  the  Design  and  Evaluation  of  the  Instruc- 
tor's Station  in  Training  Equipment.  WADC  Technical  Report  54- 
564.  American  Institute  for  Research,  Pittsburgh,  Pennsylvania, 
1954. 

Many  persons  have  come  to  identify  "training  device"  with  "simulator." 
But  this  report  deals  with  training  devices  of  which  "simulators"  are  only 
one  type  or  class.  Treatment  is  also  given  to  materials  ranging  through 
demonstration  charts,  mock-ups,  and  synthetic  training  devices  which  are 
less  than  the  extreme  of  the  engineer's  art. 

Important  as  it  is  to  be  able  to  decide  what  should  go  into  a simu- 
lator, it  is  equally  important  to  decide  whether  a simulator  is  necessary, 
and  why  a simpler  device  would  not  be  just  as  effective  from  a training 
standpoint. 

In  this  report,  therefore,  the  term  "training  device"  will  be  used 
in  the  most  general  way  to  include  any  fabricated  object,  ranging  from 
wall  chart  to  simulator,  used  for  a training  program.  This  usage  vastly 
simplifies  the  discussion. 

This  report  omits  discussion  of  one  important  dilemma  which  the 
designer  of  training  equipment  frequently  must  face.  This  is  the  problem 
of  acceptability  by  the  using  agency  where  there  may  be  pre-existing 
attitudes  contrary  to  the  designer's  conviction  of  what  is  appropriate. 

The  customer  may  often  have  excellent  reasons  for  his  attitudes  ana  even 
for  his  prejudices.  But  a policy  of  "the  customer  is  always  right,"  even 
when  it  is  against  the  customer's  best  interest,  sets  up  a vicious  circle 
which  can  be  broken  only  when  all  parties  can  cooperate  on  a common  basis 
of  information.  It  is  hoped  that  some  of  the  principles  in  various  of 
the  human  engineering  guides  can  help  establish  such  a basis  for  coopera- 
tion. 


It  would  be  unreasonable  to  expect  a designer  of  equipment  to  require 
no  more  than  a report  such  as  this  in  order  to  design  successful  training 
devices.  Not  only  a great  deal  of  specialized  engineering  know-how  but 
of  psychological  know-how  must  be  brought  to  bear  in  the  invention  of 
training  equipment  which  is  to  be  both  efficient  and  economical.  It  is 
a major  purpose  of  this  report,  however,  to  facilitate  team  work  involv- 
ing engineering  knowledge  and  skill  and  that  body  of  knowledge  and  skill 
which  deals  with  human  behavior  as  modifiable  by  training  and  experience. 
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ft  CHAPTER  I.  SOME  PRINCIPAL  CONCEP1S  IN  LEARNING  AND 

TRANSFER  OF  TRAINING 


WE  WILL  HAVE  TO  BEGIN  WITH  SCWE  OVERSIMPLIFICATIONS  1 Hasan  learning 
deals  with  such  a complex  of  behavior  variables,  that  even  to  define  learn- 
ing is  to  enter  controversial  grounds.  So  let  us  start  with  a somewhat  sim- 
plified definition  and  bear  in  Bind  that  this  definition  will  include  aost  of 
the  other  processes  identified  and  defined  in  the  rest  of  thiB  chapter. 


Definition 

Learning  is  the  process  whereby  new  or  different  responses  cone  to  be 
cade  to  given  stimuli  as  a function  of  practice.  In  practical  situations 
we  can  identify  the  effects  of  learning  when  we  know  "vhat  the  person  is  try- 
ing to  do." 


This  definition  may  be  roughly  translated  as  follows!  Learning 
is  equivalent  to  the  process  of  reprograming  the  human  computer 
mechanism  so  that  a given  sensory  input  A which  has  produced  out- 
put response  X now  produces  response  T.  The  term  "different"  in 
both  sentences  above  should  include!  "making  more  reliable  a 
previous  response  output,  such  as  X,  to  the  given  stimulus  input. " 


Practice 

INPUT 

HUMAN 

OUTPUT 

Trial  1 

Stimulus  A 

Response  X 

Practice 
Trial  100 

Stimulus  A 

Response  T 

Response  Differentiation 


This  first  general  model  is  an  oversimplification  because  practice  may 
develop  capacities  or  mechanisms  for  response  output.  Example!  Learning 
to  make  to  a signal  rapid  trigger  responses  with  the  third  finger  rather 


"1" 
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Principal  concepts 


than  with  either  the  third  or  fourth  finger.  This  idea,  which  we  call  Re- 
sponse Differentiation,  can  be  shown  as  follows: 


Practico 
Trial  1-5 


Practice 
Trials  100-105 


INPUT 


Stimulus  A 


HUMAN 


OUTPUT 
Response  X 


\ Resnonae  Y 


Stimulus  A 


Response  X 


(not  Response  Y 


RESPONSE  DIFFERENTIATION 


Response  Patterning 


But  the  learning  process  can  also  result  in  putting  responses  together 
into  patterns  occurring  either  at  the  same  time  or  occurring  as  a temporal 
series!  fhese  two  types  of  patterning,  which  are  shown  in  the  following  dia- 
grams, are  called  Response  Coordination  and  Response  Serialization. 


Practice 
Trials  1-10 


Practice 
Trials  100- 
110 


INPUT 


HUMAN 


OUTPUT 


Simultaneous  Response 
Pattern  to  A 


RESPONSE  COORDINATION 
(Simultaneous  Response  Patterning) 

Example:  Learning  to  move  stick,  rudder  bar 
and  throttle  together  in  coordinated  flight. 
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INPUT  HUMAN  OUTPUT 


Trlnln  Stimulus  A 

Response 

100-110 

Canponent  X 

RESPONSE  SERIALIZATION 

Response 


(Sequential  Response  Patterning) 


Response 

Component""? 


Example:  Recall  of  a memorized  procedure*  actual 
performance  of  a veil-practiced  starting  procedure. 


ni 

i 


Actually,  of  course,  changes  in  the  "computer  mechanis!a,,  accompany  changes 
in  response  patterning,  and  may  be  principally  responsible  foi  eliciting  the 
response  pattern  when  it  becomes  a part  of  the  response  repertory  of  the 
trainee. 


(The  reader  is  again  reminded  that  because  of  the  complexity  of  the  in- 
teractions within  the  human  mechanism  in  action,  diagrammatic  model*-  should 
always  be  regarded  as  scientific  fictions.) 


if 

] 

,1 


{ 
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Stimulus  Differentiation 


Let  us  now  turn  to  the  input  portion  of  the  system.  Effects  of  learning 
processes  may  show  up  as  systematic  differences  in  the  decoding  or  encoding 
of  messages  presented  as  inputs.  Thus: 


INPUT 


HOMAN 


OUTPUT 


Practice 
Trials  1-5 


Practice 
Trials  100- 
105 


Let  us  call  this  process  STIMULUS  DIFIERENTIATION : differential  respon- 
ses .are  learned  to  different  stimuli  to  whioh  similar  responses  were  made 
earlier. 


Example:  Learning  to  distinguish  a MIG-15 
from"  an  F-86E  Sabre-jet  in  the  air. 


Stimulus  Generalization 


INPUT 

/ 

HUMAN 

OUTPUT 

{ Stimulus  A 

Response  X 

Practice  i 

Trials  1-10 


Practice 


Stimulus  B 

Response  Y 

Stimulus  C 

Response  Z 

Stimulus  A 

Response  X 

Stimulus  B 

Response  X 
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Let  us  call  tills  process  STIMULUS  GENERALIZATION:  siallar  responses 
are  performed  to  a variety  of  stimuli  which  are  physically  dissimilar. 

Example:  Interpreting  code  messages  in  the  seme  way  irrespective 
of  the  pitch  of  the  carrying  tone. 

Stimulus  generalisation  as  a process  has  its  effect  in  "transfer  of 
training."  The  validity  (succees)  of  a training  program  depends  on  the  general- 
isation or  transfer  of  responses  from  the  synthetic  situation  (or  synthetic 
stimuli)  to  the  operational  situation  (stimuli). 

Stimulus  generalisation  may  also  come  about  through  the  learning  pro- 
cess. For  eocample,  a pilot  who  reads  altitude  inforsat5.cn  from  the  convention- 
al three-needle  altimeter  can  learn  to  get  equivalent  information  from  a count- 
er-single-ne ©die- type  altimeter.  To  this  pilot,  after  sufficient  practice, 
both  instruments  are  functionally  similar  with  respect  to  the  "altitude  response" 
he  makes. 

Much  of  the  discussion  in  later  chapters  will  pivot  around  the  conditions 
which  produce  transfer  of  training  and  transfer  of  what  kind  of  training  or  re- 
sponse repertories.  But  in  order  to  state  principles  we  will  first  have  to  de- 
fine a few  more  of  the  required  terms. 


Stimulus  Patterning 

Practical  situations  invariably  require  perceptual  response  to  configur- 
ations of  stimuli  in  time  and  space.  The  car  driver  responds  to  a configur- 
ation compounded  from  various  instrument  readings  and  complex  data  about  the 
environment  such  as  road  conditions,  nearness  of  other  cars,  band  in  the  road 
ahead,  and  so  forth. 

A perceptual  response  is  therefore  inevitably  a response  to  pattern,  of 
stimulus  inputs.  A variety  of  patterns  of  stimulus  inputs  may  become, 
through  training,  functionally  equivalent  in  producing  some  one  response  or 
response  group.  They  may  become  equivalent  by  virtue  of  pattern  in  space  or 
time  or  both.  In  training,  an  example  is  the  learning  of  a set  of  verbal 
procedures  to  a photograph  of  equipment.  The  trainee  does  not  mistake  the 
photograph  for  the  real  objects  depicted.  Nevertheless,  he  may  transfer 
(generalise)  his  verbal  procedures  quite  substantially  to  the  work  situation 
filled  with  the  actual  objects  in  the  photograph. 

¥e  are  also  able  to  recognize  a tune  played  in  one  key  when  it  is  played 
in  another  key;  a telegraphic  message  when  delivered  through  the  ear  by  head- 
phones or  through  the  eye  by  blinkers. 

V/hen  there  is  a denand  for  transfer  of  training  to  superficially  differ- 
ent situations,  some  modes  of  training  which  emphasize  response  to  patterns 
of  cues  will  be  more  effective  than  others  which  emphasize  specific  response 
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to  specific  cues.  Training  which  directs  response  to  the  total  configura- 
tion of  the  work  environment  will  therefore  enable  the  trainee  to  cope  with 
wider  varieties  of  work  situation,  including  transfer  from  synthetic  to  actu- 
al work  environments. 

Through  practice,  certain  responses  can  become  at  least  partial  stimuli 
which  trigger  off  others  in  response  series  or  groups.  When  such  response  se- 
quences are  active,  they  beccme  part  of  the  total  stimulus  context  to  which 
the  human  responds;  this  associative  response  context  also  holps  us  understand 
how  human  beings  can  resist  irrelevant  stimulus  inputs  during  an  ongoing  task 
activity.  Associative  response  context  also  helps  acoount  for  a pilot  being 
able  to  have  one  set  of  habits  appropriate  to  flying  one  type  airplane  and 
another  set  of  habits  appropriate  to  flying  another  type  airplane,  even  though 
the  display  and  control  configurations  may  be  quite  different.  Similarly 
person  can  know  and  speak  several  quite  different  languages  without  one  inter- 
fering with  another. 


PERCEPTUAL,  IDEATIONAL  AHD  MOTOR  PROCESSES 

It  is  ocssstimes  helpful  (although  sometimes  misleading)  to  differentiate 
segments  of  the  learning  process  into  PERCEPTUAL  processes,  IDEATIONAL  or  SYM- 
BOLIC processes  and  MOTOR  processes. 


COMPUTER  MECHANISM 

SENSORY  MECHANISMI 1 RESPONSE  MECHANISM 


Ideational 

Processes 


This  distinction  is  important  because  under  some  conditions  the  ideation- 
al processes  (thinking,  reasoning)  may  become  by-passed  by  virtue  of  the  learn- 
ing process.  Such  by-passing  has  important  implications  for  learning  and  for 
capacities  for  performance,,! 

Ideational  processes  include  encoding,  i.e.  transforming  an  environmental 
stimulus  into  a symbolic  stimulus;  associative  symbolic  stimulus-response  pat- 


1*  Tha  diagrams  on  p.  12  shew  symbolic  and  non-symbolic  channels. 
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terns  stored  in  memory  (as  in  self-instructions )j  and  the  conversion  of  sym- 
bolic responses  into  response  output.  Ideational  processes  also  may  include 
"anticipatory*  systems  and  purposive  or  motive-incentive  processes. 


HOW  DO  WE  GET  THE  TRAINEE  TO  MAKE  NEW  RESPONSES? 

In  order  for  a new  response  to  be  learned  to  a stimulus  A,  it  must  be 
practiced  to  stimulus  A.  But  how  do  we  get  a new  response  to  occur  in  the 
first  place? 

In  humans,  on  important  mechanism  is  that  of  symbolic  stimulus-response. 
For  example,  a red  light  appears  on  a panel.  Ths  trainee's  hand  remains  idle 
on  uhe  control  panel.  The  instructor  tells  him  that  when  the  light  appears 
he  must  press  switch  X.  On  the  next  trial  when  the  light  appears  to  the  train- 
ee it  combines  with  the  symbolic  stimulus -response  (the  instructions)  and  the 
trainee  presses  switch  X.  He  has  made  a new  response. 

Now  let  us  go  on  to  the  situation  in  which  the  trainee  uses  the  consequen- 
ces of  his  response  to  improve  his  response  output. 

Informative  Feedback 


The  following  diagram  contains  the  elements  of  a feedback  system. 
INPUT  HUMAN  OUTPUT 


Stimulus 



Response 

1 

Jr— 

i 

l 

Information  about  Response 

FEEDBACK  CHANNELS 

There  are  two  major  types  of  informative  feedback.  The  distinction  is  based 
not  necessarily  on  any  characteristic  of  the  signal  itself,  but  in  the  way  it  is 
used  by  toe  trainee*  ^ 

Learning  feedback t Let  us  presume  voluntary  or  "ideationally"  guided  be- 
havior. A feedback  system  presents  the  trainee  with  his  response  output  com- 
pared with  what  that  output  should  have  bs®n. 
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INPUT 


HUMAN 


output 


Tine  1 


Stimulus  A 
recurs 


^ Computer 


Time  2 


Haaim 

Response  7*’ 


Machine 
Response  I 


Information  about 
Stimulus  A 
Human  Response  X 
Machine  Response  Z 


LEARNING  FEEDBACK 
(Feedback  acting  in  time) 

In  this  case,  the  error  feedback  results  in  a change  in  the  computer  (or 
as  stored  information)  so  that  when  stimulus  A recurs  on  a later  date,  a some- 
what different  signal  will  be  issued  from  the  computer  to  the  response  mechan- 
ism. That  is,  in  effect,  reprograming  the  computer.  In  this  way  a man  learns 
to  aim  a gun  better  on  the  tenun  day  of  practice  than  on  the  first  if  he  sees 
the  size  and  direction  of  Ms  misses. 

Let  us  call  feedback  which  acts  in  this  way  "learning  feedback."  Another 
term  for  this  activity  is  knowledge  of  results.  But  usage  also  gives  another 
moaning  to  "knowledge  of  results^  as  our  'herb  example  will  show. 

The  effects  o;  learning  feedback  are  not  revealed  until  another  occasion 
on  which  INPUT  A occurs  as  such,  to  this  occasion,  INPUT  A produces  a re- 
sponse compounded  of  the  response  tendency  and  the  "correction  factor"  intro- 
duced into  the  computer  by  the  learning  feedback.  The  resultant  is  response 
I which,  we  hope,  has  less  error  in  it  than  did  the  former  response  X. 

(it  is  as  if  the  mechanism  said:  "When  Stimulus  A occured  I should  have 

made  Response  Y instead  of  Response  Z and  the  nest  time  Stimulus  A oc- 
curs I will  respond  with  I.") 

It  follows  that  the  more  efficiently  the  learning  feedback  informs  the 
trainee  about  his  response,  the  more  readily  he  can  learn.  He  should  be  in- 
formed only  about  what  he  can  control.  We  conclude  that  for  efficiency  in 
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training],  learning  feedback  should  not  be  contaminated  by  machine  variable 
error,  jn  this  respect,  as  in  others,  the  synthetic  training  situation  may 
be  more  efficient  then  the  operational  training  situation. 

Action  feedback!  Consider  a tracking  task  such  as  pointing  a gun  at  a 
etl  iffie  difference  (error)  between  momentary  point  of  aim  and  tar- 


moving 
get  posijt: 
a stimul 


SF5 


ion  is  a signal  fed  back  to  the  operator  who  uses  this  information  as 
is  for  succeeding  momentary  corrective  adjustments.  Because  these 
adjustments  direct  aotion  uo  the  error  signal  by  utilizing  present  programs  in 
the  human  computer,  let  us  call  this  information  "aotion  feedback." 


INPUT 


HUMAN 


OUTPUT 


Action  Feedhack 


Stimulus 


_ JLJ: 


Stimulus 


If 


Solid  line 
Dashed  line 


Time  1 
Time  2 


Response 


Response  I 


ACTION  FEEDBACK 


(Herb  it  is  as  if  the  computer  said:  "Response  X must  now  be  changed  into 
Response  I in  order  to  reduce  the  error  now  presented.") 

Note  that  action  feedback  does  not  change  the  characteristic  of  the  compu- 
ter (by  definition)  but  learning  feedback  (by  definition)  does  tend  to  change 
the  characteristic  of  the  human  computer.  The  same  external  channel  of  error 
information  may  carry  both  action  feedback  and  learning  feedback.  Whether  the 
feedback  signal  acts  in  one  capacity  or  the  other,  or  both,  is  in  part  due  to 
the  "set,"  "intent"  or  motivation  of  the  trainee  operator. 


Response  to  available  feedback  is  always  selective  at  the  perceptual,  idea- 
tional and  motor  levels.  Effective  training  guides  the  selective  tendencies  of 
the  trained,  according  to  his  momentary  capacity  for  handling  pattern  of  learn- 
ing feedback  plus  action  feedback. 


Many 


persons,  including  specialists,  use  the  term  "knowledge  of  results" 
to  include  either  or  both  learning  feedback  and  action  feedback.  In  fact,  the 
distinction  between  action  and  learning  feedback  has  rarely  been  recognized. 
Because  of  the  important  implications  to  training  of  the  difference  between 
these  two  terms , however,  we  will  use  them  in  this  report. 
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Changing  Beaponao  Through  Anticipation 

Lat  ua  assume  the  path  of  a target  has  a given  characteristic  each  as 
that  of  a sine  (regular)  wave.  On  trial  1 the  operator  will  be  one  reaction- 
time behind  the  target. 


Notice  the  lag  of  the  cursor  response,  and  the  resulting  over-travel. 
With  continued  practice  he  will  learn  to  anticipate  the  course  of  a target 
upon  identifying  that  target  and  getting  into  phase  with  it. 


Now  notice  that  after  the  initial  latent  period,  the  cursor  catches  up 
to  the  target  and  instead  of  merely  following  it,  keeps  up  with  it*  It  is 
convenient  to  think  of  this  performance  as  showing  anticipation  of  what  the 
target  is  going  to  do  next.  The  him  an  computer  which  guides  the  cursor 
(which  may  be  the  husen  hand  itself)  has  acquired  the  same  program  as  that  of 
the  targets  generator,  plus  a Thasdjsg-in  mechanism.  Ths  pk&sing-in  mechanism 
keeps  the  cursor  track  from  remaining  one  reaction  time  behind  the  target 
track. 


Note  that  learning  may  reach  the  point  at  whir  _ no  more  than  the  target 
signal  at  time  0 (zero)  need  be  shown.  From  this  starting-signal,  the  human 
computer  may  trace  out  the  proper  space-time  path  without  further  data  from 
the  tarsal  during  the  mission*  This  is  the  analogy  in  tracking  behavior  of 
the  conditioned  response* 
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Changing  Response  Through  Stimulus  Combinations 

Response  to  a given  stimulus  may  change  when  that  stimulus  becomes  a com- 
ponent in  new  patterns  of  stimulation.  Thus  the  contours  of  a battleship  at 
sea  may  become  unrecognizable  when  it  Is  camouflaged,  (later  we  shall  see  that 
there  is  not  a one-to-one  relationship  between  response  to  a physical  stimulus 
in  one  context  and  to  that  physical  stimulus  in  another  context  of  events.) 


INPUT 

HUMAN  OUTPUT 

1.  Original  Stimulus  A 

Response  X 

Habit  System 

Stimulus  B 

“““ 

Response  X 

2.  Intervening 

• 

Practice  Conditions 


3.  Later 

Habit  System 


Stimulus  A 
Stimulus  B 


Response 
X I ? 


CHANGING  RESPONSE  THROUGH  STIMULUS  COMBINATIONS 


The  brace  in  the  computer  symbolizes  a combining  activity. 

Speaking  in  general  terms,  stimulus  contexts  include  one  or  more  of  the 
following  factors; 

What  other  physical  stimuli  are  present  at  the  same  time 
What  physical  stimuli  preceded  the  stimulus 
The  prevailing  "motivation"  of  the  trainee 
"attitude" 

"set  to  respond" 
etc. 

The  terms  motivation,  attitude,  set  to  respond,  put  huge  burdens  on  keep- 
ing our  model  simple.  Let  us  for  the  time  being  recognize  their  importance  to 
learning  processes  without  trying  to  get  them  into  our  diagram. 

Incidentally,  practice  to  suitable  combinations  of  stimulus  A and  B may 
lead  to  both  responses  X ami  X being  made  to  either  stimulus  A or  B. 
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Training  usually  refers  to  highly  complex  behaviors  acquired  more  or  less 
through  voluntary  intent  of  the  learner.  We  oust  therefore  include  as  perhaps 
the  most  significant  determiners  of  new  responses  to  environmental  inputs, 
the  capacity  for  symbolic  stimulus -response  activity.  Thus  the  driver  may 
think  or  say  to  himself:  "This  map  shows  a short-cut j therefore,  when  I cone 
to  the  next  corner  (new  symbolic  stimulus)  I will  turn  right  (new  symbolic 
response)" instead  of  turning  left  (old  symbolic  response).11 


INPUT 


HUMAN 


OUTPUT 


Stimulus 
Input  m 


Conversion 

Symbolic 

Conversion 

to  Symbolic 

^ 
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to  Motor  _ 

Stimulus 

Formation 

Response 

Response  Response 
Mechanism  Output 


-Non-Symbolic  Channel 


CHANGING  RESPONSE  THROUGH  SYMBOLIC  IDEATIONAL  ACTIVITY 

Notice,  however,  that  the  driver  was  using  stored  information 
when  he  recalled  the  short-cut  shown  on  the  map.  tet  us  add  this  as 
another  feature  to  our  diagram: 


INPUT 


HUMAN 


OUTPUT 


Stimuli 

Input 


Conversion 

Symbolic 

Conversion 

to  Symbolic 

-e? 

Response 
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to  Motor 

Stimulus 
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Response 

Stored  Information 
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Response 

Output 


Non-Symbolic  Channel  


SYMBOLIC  BEHAVIOR  UTILIZING  STORED  INFORMATION 
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Symbolizing  processes  imply  decoding  and  encoding  capacities  of  the  cen- 
tral mechanism.  That  is,  in  order  for  symbolic  activity  to  occur,  there  rust 
be  symbolic  representations  of  stimulus  inputs  and  output  responses  compatible 
with  each  other  and  with  stored  information.  This  matter  of  code  compatibili- 
ty is  an  important  area  in  the  study  of  human  behavior. 

Note,  however,  that  we  have  vastly  expanded  the  learning-capacities  of 
the  mechanism  by  introducing  symbolic  behavior! 

The  following  kinds  of  *•  information”  can  be  and  are  symbolized  in  the  hu- 
man computer  systems 

Stimulus  objects,  stimulus  relations,  stimulus  classes 
Responses  and  "sets  to  respond’1 
Response  feedback 

Motivations,  incentives  and  goal  conditions 

Symbolic  proceeses  or  coding  systems  include  words,  images  and  concept- 
ual processes  few  of  which  can  be  precisely  correlated  with  objective  items  or 
sequences.  For  our  purposes,  it  is  more  or  less  sufficient  to  know  how  to  con- 
trol the  external  conditions  under  which  theso  processes  can  be  induced  so  as 
to  affect  learning  and  performance. 

It  will  be  convenient  to  refer  to  these  symbolizing  processes  which  occur 
inside  the  human  "black  box"  as  mediating  stimulus -response  systems  or  activi- 
ties. They  "mediate"  stimuli  from  the  environment  and  responses  performed  on 
the  environment.  Such  processes  have  payoff  significance  in  problem-solving 
activities,  in  the  giving  of  self-instructions,  and  in  the  making  of  decisions. 
All  three  of  these  activities  are  probably  closely  inter-related. 

Ideational  stimulus -response  systems  have  an  important  role  in  transfer  of 
training  in  that  they  can  make  superficially  different  stimuli  similar.  We  can 
8 ay  or  think  "One  dollar"  whether  we  see  or  think  of  a hundred  pennies,  a dol- 
lar bill  or  the  figures  01.00  printed  on  a check.  Through  suitable  use  of  idea- 
tional capacities  we  can-  speed  up  the  learning  of  complex  motor  rospont  s,  and 
amplify  the  range  of  stimuli  to  which  training  can  be  generalised. 

Changing  Response  Through  Habituation,  Sensory  Adaptation,  Fatigue 

Within  limits,  the  magnitude  of  response  to  a constant  value  continuing 
stimulus  input  tends  to  diminish.  We  plunge  into  water  which  seems  cold  at 
first  but  after  ten  minutes  the  water's  temperature  seems  neutral,  unless  it 
is  below  certain  limits.  NoiBes  on  a telephone,  which  at  first  may  be  disturb- 
ing to  a verbal  message,  become  less  noticeable  after  a few  minutes  conversa- 
tion. The  more  constant  and  less  variable  the  background  portion  of  the  stimu- 
lus, the  more  rapidly  it  ceases  to  distract  — that  is  to  elicit  disturbing  or 
other  responses. 

When  this  phenomenon  occurs  within  one  session  of  activity  it  is  called 
"sensory  adaptation”  cap  "negative  adaptation."  When  it  happens  from  one  sess- 
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ion  of  activity  to  another  it  is  called  "habituation." 

Seme  writers  have  related  sensory  adaptation  and  habituation  to  aenaory 
fatigue,  but  wo  need  not  be  concerned  with  the  problems  raised  by  this  issue. 

Two  important  conclusions  can  bo  recognized  from  these  phenomenal 

1.  An  input  will  tend  to  aot  as  a stimulus  (that  is,  produce  re- 
sponse) when  it  is  a change  in  some  previously  existing  state 
of  inputs.  Its  strength  as  a stimulus  will  be  related  to  its 
rate,  as  well  as  absolute  magnitude,  of  change.  The  organlfs 
is  rarely  placid  to  environmental  inputs  of  the  moment,  how- 
everj  it  contributes  to  the  stimulus  conditions  through  idea- 
tional and  motivational  (purposive)  activity. 

2.  Response  outputs  are  modified  through  the  organic  phenomena  of 
sensory  adaptation  and  habituation. 


Note  to  the  academic  reader:  Vexation  is  caused  by  the  frequent 
ambiguity  of  the  term  "stimulus."  Let  us  agree  that  an  environ- 
mental stimulus  input  be  thought. of  as  a stimulus  only  when  it 
elicits  an  actual  or  hypothetical  response  in  "itie  Inman  system. 

This1  distinction  "has  substantial  'bearing  on  our  later  distinction 
between  engineering  simulation  and  psychological  simulation  (see 
Chapter  II,  pago  2). 

Objectively,  fatigue  is  the  roduction  of  the  likelihood  of  a given  re- 
sponse to  a given  stimulus  configuration  as  a function  of  the  rate,  frequency 
and  recency  of  making  that  response.  This  observation  seems  to  hold  for  idea- 
tional as  well  as  motor  processes.  As  a consequence  of  practice,  learning  ef- 
fects tend  to  be  obscured  by  fatigue  effects ; and  the  learning  process  may,  in 
some  respects,  interact  with  the  fatigue  process.  In  general,  learning  ef- 
fects are  more  persistent  than  fatigue  effects.  Nevertheless,  the  interact- 
ion of  the  learning  and  fatigue  processes  suggests  optimal  periods  of  pract- 
ice versus  rest  for  any  given  type  of  task.  Research  into  "massed  versus  dis- 
tributed practice"  confirms  this  by  showing  diminishing  return?;  for  time  spent, 
beyond  a given  point,  in  continuous  practice  before  a rest  interval.  Optimal 
ratios  of  practice  to  rest,  and  times  for  rest,  are  related  not  only  to  kind 
of  task,  but  to  incentive  conditions  and  stage  of  learning. 

In  any  event,  sensory  adaptation,  habituation  and  fatigue  can  account 
for  changes  in  response.  To  the  extent  that  these  changes  in  responses  are 
consistent,  they  may  become  part  of  the  total  pattern  of  response. 

Motivation 

Our  treatment  of  motivation  will  be  necessarily  brief  and  cavalier.  Let 
us  presume  (often  rashly)  that  the  trainee  wants  to  perform  the  job  reliably 
although  actually  part  of  the  function  of  training  may  be  to  induce  a desire 
to  learn  the  skills  required. 
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We  can  think  of  motivation  as  performing  two  main  functions  in  the  learn- 
ing^odeli  an  amplifier  of  response  tendencies  and  a filter  of  response  feod- 


INPUT 


HD3M 


OUTPUT 


Motivation  energizes  the  learning-action  system,  and  energizes  it  select- 
Motivations  may  be  highly  specific,  and  are  learnsble.  They  are  iden- 
tified through  incentives.  An  incentive  is  some  goal  (or  null.)  position  se- 
lected. out  of  a number  of  possible  goals  or  goal  variables. 

Jbg&QDlas  A pilot  trainee  is  motivated  to  reduce  azimuth  error.  He 
does  so,  but  neglects  error  in  elevation  and  tracking  smoothness. 

Motivations  are  also  incorporated  in  attitudes  of  acceptance  or  rejection 
of  objects  and  situations.  Gross  attitudes  of  rejection  interfere  with  learn- 
ing and  transfer  of  training. 
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SUMMARY 

Learning  is  the  process  whereby  new  or  different  responses  are  made  to 
given  stimuli  as  a consoquence  of  practice.  Practice  is  defined  as  making  a 
given  response  to  a given  stimulus.  Practice  may  develop  new  capacities  for 
responso  output  through  response  differentiation,  response  coordination,  and 
response  serialization.  On  the  stimulus  input  side,  suitable  modes  of  prac- 
tice develop  stimulus  differentiations,  responses  to  complex  patterns  of 
stimulus  inputs,  and  transfer  of  responses. 

For  convenience  in  the  planning  of  training  it  is  helpful  to  distinguish 
between  perceptual,  ideatiomd  (symbolic),  and  mote*-  pi  bosses. 

In  order  to  learn  a new  response,  it  must  be  made  and  mado  to  the  appro- 
priate stimulus.  Processes  by  which  new  responses  can  be  induced  to  occur 
include  (a)  the  development  of  anticipations,  (b)  stimulus  combinations,  (c) 
symbolic  activity,  (d)  sensory  adaptation  and  habituation  — or  combinations 
of  these  processes. 

Essential  to  improvement  in  skill,  however,  is  information  about  the  re- 
sults of  the  output  response.  When  this  feedback  is  carried  over  into  a re- 
currence of  the  original  stimulus  so  as  to  reduce  error  on  the  later  trial, 
the  signal  is  called  "learning  feedback."  When  an  error  signal  is  used  to 
modify  the  response  in  progress'^  is  called  "action  feedback."  Much  effic- 
iency in  learning  depends  on  the  availability  (arid  use  by  the  trainee)  of  ap- 
propriate learning  feedback. 

Motivation  acts  as  an  amplifier  to  the  entire  system  and  a selective  fil- 
ter to  feedback  information. 


TRANSFER  OF  TRAINING 

When  a response  is  made  to  a stimulus  it  tends  to  get  learned  to  that 
stimulus.  When  that  stimulus  recurs,  the  response  will  tend,  by  the  degree  it 
has  been  learned,  to  recur  also.  (In  one  sense,  the  response  is  "transferred" 
from  one  occurrence  of  the  stimulus  to  a later  occurrence  of  that  stimulus.) 

But  a set  of  stimulus  events  is  never  precisely  repeated.  It  is  therefore  dan- 
gerous to  speak  of  "the"  stimulus  as  if  we  ever  could  specify  all  the  stimula- 
ting conditions  which,  at  any  moment,  are  associated  with  a given  response. 
Nevertheless,  when  responses  recur  to  stimulus  events  which  were  similar  to  the 
earlier  stimulus,  He  speak  of  memory  or  habit. 

But  we  may  change  the  original  stimulus  in  a variety  cf  ways  and  find  in 
some  cases  that  the  responses,  in  some  degree,  transfer  to  the  changed  stimulus. 
We  will  find  cases  of  high  transfer  of  response  to  apparently  very  dissimilar 
situations.  In  other  cases  we  will  find  that  even  rather  small  changes  in  sti- 
mulus conditions  produce  little  transfer  of  an  earlier  response.  If  we  can  be 
specific  about  which  conditions  make  fr*  greater  and  which  for  lesser  transfer 
of  response  (or  training)  we  can  obviously  b3  economical  in  training.  This 
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will  be  especially  tho  case  if  the  operational  conditions  involve  stimuli  very 
expensive  to  produce,  such  as  an  airplane,  guns  and  aerial  targets. 

Transfer  of  training  is  transfer  of  responses  from  one  set  of  stimulus 
conditions  to  another  set  of  stimulus  conditions.  Training  devices  and  class- 
room studies  set  up  synthetic  stimulus  situations.  We  hope  that  the  responses 
learned  to  these  synthetic  aituations  will*  (a)  aid  in  learning  operational 
responses;  (b)  tr ana fee  positively  to  operational  situations;  (c)  cover  oper- 
ational situations. 

An  additional  note.  We  have  “transfer  of  training"  even  when  the 
transferred  habits  are  inappropriate  to  the  new  situation.  In  such  cases, 
the  habits  may  interfere  with  the  learning  or  execution  of  the  correct 
response  in  the  new  situation.  This  is  usually  the  problem  -in  transition 
training  where  similar  sets  of  stimuli  now  demand  different  (and  erven  re- 
versed) response  patterns.  This  "transfer  of  training"  has  negative  or 
interference  effects.  Obviously,  training  aliould  be  directed  not  only 
towards  transfer  of  habits  to  operations,  but  transfer  of  appropriate 
habits  and  the  avoidance  of  inappropriate  habits  and  attitudes.  The 
transfer  of  these  latter  may  require  unlearning  during  operations  which 
can  be  hazardous  as  well  as  expensive. 


Perceptual  Sonsitivity  And  Stimulus  Variation 

Human  receptor  channels,  like  other  receivers,  have  limitations  in  sensi- 
tivity. That  is,  for  any  stimulus  input,  there  will  be  ranges  through  which 
physical  differences  will  not  be  matched  with  behavioral  differences.  These 
ranges  of  insensitivity  will  differ  for  different  modes  of  response. 

For  ev.unple,  a person  may  readily  distinguish  a white  from  a yellowish 
illumination  in  stepping  from  one  room  to  another;  if  however,  you  ask  him  to 
read  a passage  of  prose  first  in  one  room  and  then  in  the  other,  no  difference 
in  reading  may  be  discovered  within  the  criteria  established  for  reading  effect- 
iveness. 

Similarly,  seasoned  pilots  may  b9  able  to  tell  us  that  the  feel  of  the 
controls  in  a simulator  differs  from  the  feel  of  the  corresponding  controls 
in  their  aircraft.  Their  performance  of  maneuvers  in  tho  simulator  may,  how- 
ever, reveal  no  measurable  difference  of  response  output  within  the  operation- 
al criteria  established  for  that  performance.  On  the  other  hand,  the  converse 
may  be  trues  differences  between  stimuli  which  an  operator  may  fail  to  dis- 
criminate at  the  verbal  level  may  show  up  in  job  belmvior. 

Stimulus  validity  should  therefor©  be  established  on  the  basis  of  what  the 
operator  has  to  do  on  the  job,  rather  than  irrelevant  responses. 

Economy  and  feasibility  require  that  the  training  device  copy  the  opera- 
tional equipment  only  within  the  range  of  stimulus  inputs  which  produce,  or  are 
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likely  to  produce,  a demonstrable  change  in  essential  job  behaviors.  Para- 
meters and  formulas  are  not  now,  or  in  the  noar  future  likely  to  bo,  avail- 
able for  estimating  "transfer  sensitivity"  to  stimulus  differences.  The 
educated  guesses  of  specialists  in  human  learning  will  probably  be  the  best 
basis  for  decisions. 


Transfer  And  Type  Of  Task 

It  has  already  been  suggested  that  some  types  of  activity  transfer  to  a 
wider  range  of  stimuli  than  others.  For  example,  ideational  forms  of  behav- 
ior tend  tc  be  less  "stimulus-bound"  to  specific  environmental  cues  than  most 
kinds  of  manual  or  motor  response. 

We  can  also  expect  differences  in  transferability  among  types  of  motor 
performance.  Let  us  distinguish  between  procedural  tasks  and  continuous  feed- 
back tasks. 

Procedural  or  discontinuous  tasks  are  tasks  in  which  separate,  mainly  all- 
or-none,  responses  are  made  to  given  cues.  Following  a set  of  printed  (or  re- 
membered) instructions  in  pressing  a series  of  switches  to  a sequence  of  sig- 
nals would  be  an  example.  Procedures  are  often  verbally  mediated.  When  they 
are,  they  may  transfer  to  widely  different  stimulus  inputs.  With  high  degrees 
of  practice,  of  course,  verbal  mediation  may  tend  to  drop  out  and  the  responses 
become  automatic.  When  they  do,  we  will  get  somewhat  less  generalization  to 
different  stimuli. 

Every  job  has  a large  number  of  procedural  portions,  many  of  which  can  be 
learned  with  relatively  crude  physical  copies  or  even  symbolic  representations 
and  mock-ups. 

Continuous  feedback  tasks  are  also  known  as  tracking  tasks.  Performance 
involves  a continuously  changing  response  to  a continuously  changing  stimulus 
input.  ("Continuous"  refers  to  the  maximum  continuity  capable  of  the  human 
operator  as  a system.)  The  response  is  a continuous  adjustment  by  tha  operator 
to  a changing  error  signal.  The  error  signal  results  from  the  operator's 
comparison  of  the  desired  output  with  the  changing  output  signal  which  is  fed 
back  to  him. 

In  general,  transfer  of  training  in  tracking  tasks  tends  to  be  more  re- 
stricted than  it  is  in  procedural  tasks,  especially  if  time  changes  are  involved. 
That  is,  the  operator  learns  sets  of  anticipations  of  signal  inputs,  and  when 
these  anticipations  do  not  conform  tc  the  sequences  and  patterns  of  cues  in  the 
new  situation,  the  old  responses  are  inadequate. 

This  topic  is  given  expanded  treatment  in  Chapter  III,  Stage  of  Learning 
and  Physical  Simulation,  p,  31. 

The  statements  in  this  chapter  are  intended  to  present  an  introductory 
overview  to  the  concepts  in  the  following  chapters  which  will  apply  these  de- 
finitions to  problems  more  directly  connected  with  equipment  design. 
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i 

C 


INTRODUCTION 


Training  consists  of  the  acquisition  of  skill  in  using  equipment  in 
operational  environments.  Let  us  call  any  set  of  environmental  conditions 
a "program." 

A simple  example  may  clarify  the  moaning  of  the  terms.  We  wish  to 
train  a person  to  drive  a car.  This  statement  may  imply  programs  but  it  does 
not  specify  them.  r.*iving  a car  on  the  open  road;  in  dense  city  traffic; 
two-lane,  truck-cluttered  roads;  icy  roads;  night  illumination  — all  these 
are  program  variables  in  driving. 

The  problem  of  simulation  must  take  into  account  the  representation  of 
programs  as  well  as  the  representation  of  the  equipment  and  its  functioning 
to  the  trainee. 


PSYCHOLOGICAL  SIMUIATION  VffiSUS  ENGINEERING  SIMUIATION 

Simulation  is  not  all  or  none;  it  is  a matter  of  degree.  Perfect 
simulation  consists  of  the  job  itself  performed  on  operational  equipment  in 
the  context  of  all  its  actual  programs.  The  fact  that  a situation  is 
synthetic  in  itself  is  less  than  complete  simulation.  Items,  functions,  and 
programs  are  selected  for  copying,  and  each  for  copying  to  varying  degrees 
of  physical  reproduction. 

We  should  therefore  not  speak  of  pnysical  simulation  as  if  it  were  an 
object  which  must  be  either  present  or  absent.  Even  in  physical  terms, 
simulation  is  a matter  of  degree. 


The  Training  Problem 

But  the  training  problem  is  not  a matter  of  physical  copying  of 
equipment  and  its  interactions  with  physical  replicas  of  programs.  Let  us 
give  the  name  "engineering  simulation"  to  the  copying  of  some  physical  model 
and  its  physical  properties. 

The  training  problem  is  to  provide  stimuli  so  that  responses  learned 
to  them  will  transfer  from  training  to  operations  with  little  or  no  loss . 
When  responses  learned  in  training  transfer  into  appropriate  action  in 
operations,  let  us  think  of  the  relation  of  the  synthetic  to  the  real  con- 
ditions as  "psychological  simulation. " 
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Psychological  simulation  may  be  far  removed  from  physical  realism. 

Take  an  extreme  example.  An  instructor  tells  a trainee  in  a bare  classroom 
that  when  he  enters  an  airplane  cockpit  he  should  immediately  slam  the 
cockpit  door  forcefully.  The  trainee  goeB  out  to  the  aircraft,  enters  it, 
and  slams  the  door  forcefully.  The  classroom  situation  has,  in  terms  of 
the  job  outcome,  (in  this  instance)  psychologically  simulated  the  operation- 
al^ ituation. 

In  this  case,  the  simulation  was  effected  by  symbols — words  in  sentences 
presented  to  the  trainee  who  "practiced11  the  response  of  slamming  the  door 
symbolically  to  a symbolic  stimulus. 

Tne  point  here  is  not  "Can  you  train  such  a response  in  this  way?" 

The  point  is,  "To  the  extent  that  there  is  transfer  of  response  so  that  the 
job  is  performed  correctly,  we  have  psychological  simulation." 


Interrelationship  Of  Transfer,  Economy,  And  Engineering  Simulation 

Failure  to  distinguish  between  psychological  and  engineering  simu- 
lation may  produce  costly  excesses.  The  engineer  is  charged  to  "build  a 
simulator."  At  once  he  undertakes  to  synthesize  in  hardware  all  the  physi- 
cal apparatus,  and  perhaps  one  or  two  arbitrarily  chosen "programs."  His 
limits  are  defined  only  by  budget  and  the  state  of  the  engineering  art. 

There  are  kinds  of  task  and  degrees  of  learning  which  may  profit  from  high 
degrees  of  physical  fidelity  in  many  features  of  the  task  environment. 

Other  tasks,  or  degrees  of  skill,  may  be  learned  and  transfer  quite  adequate- 
ly from  synthetic  trainers  having  relatively  little  physical  or  functional 
realism. 

At  least  from  the  standpoint  of  economy,  the  development  of  training 


devices  should  rest  on  psychological  simulation  rather  than  engineering 


simulation.  Therefore  to  the  extent  that  engineering  simulation  is  a matter 


of  selection  and  of  degree,  the  selection  of  variables  should  be  based  on 


psychological  considerations  as  to  what  will  maximize  validity  of  training. 


With  respect  to  the  psychological  simulation  of  tasks  we  will  be  con- 
cerned with  two  main  problems.  One  of  these  is  the  extent  to  which  the 
training  includes  <#hat  is  necessary  to  include.  The  other  problem  is  what 
can  be  excluded  as  irrelevant  to  the  training  problem.  Cutting  across  both 
of  these  considerat ■ on&  is  the  degree  to  which  engineering  variables  in  the 
training  situation  need  to  duplicate  the  operational  situation  for  practical 
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amounts  of  transfer  to  obtain.  Although  the  interrelationship  of  these  three 
considerations  has  ^oen  recognized,  the  independence  of  them  has  not  general- 
ly been  made  explicit. 

Once  we  recognize  that  there  are  degrees  of  both  engineering  simulation 
and  degrees  of  psychological  simulation,  we  must  realize  that  practical 
decisions  about  the  specifications  for  training  equipment  must  rest  on 
economic  and  training  objective  compromises.  As  engineering  simulation  in- 
creases, the  equipmsnt  tolerances  grow  tighter  and  the  equipment  becomes  more 
expensive  to  build  d.id  maintain. 

At  least  up  to  a point,  there  are  increments  of  training  value  (depend- 
ing on  the  kind  and  level  of  skill  which  is  being  trained)  with  increased 
engineering  simulation.  But  the  gains  in  skill  may  be  uneconomical  from  the 
standpoint  of  the  dollars  and  cents  involved  in  building  and/or  maintaining 


the  training  device. 

To  take  an  extreme  example,  we  should  not  spend  a million  dollars  to 
build  a training  de  ice  when  the  task  can  be  taught  just  as  well  on  the 
operational  device  , ->  the  ten  thousand  trainees  we  need  for  a dollar  apiece. 

The  figure  below  is  an  oversimplified  demonstration  of  the  relation- 
ships between  degree  of  engineering  simulation,  cost  of  such  simulation,  and 
the  amount  of  transfer  of  training  value  to  be  derived  from  increased  engineer- 
ing simulation.  The  figure  may  be  interpreted  roughly  as  follows.  The  line 
marked  "cost"  shows  that  as  degree  of  engineering  simulation  increases,  costs 
go  up  at  an  increasing  rate.  This  increase  in  cost  is  particularly  true  when 
not  only  the  mean  cf  some  variable  on  the  operational  equipment  is  reproduced, 
but  when  this  mean  is  reproduced  with  increasingly  small  operating  tolerances 
in  the  training  device. 

But  physical  realism  is  not  only  a matter  of  degree  with  respect  to  a 
variable,  but  in  number  and  kind  of  v iables  represented.  That  is, 
more  and  more  'ua-acteristics  of  the  operational  situation  may  be  put  into 
the  trainer,  aui  each  of  these  added  characteristics  may  increase  cost  dis- 
proportionateV  thm-gh  multiple  interacting  linkages. 

The  relationship  of  cost  to  degree  and  kind  of  simulation  can  be  rather 
readily  determined,  although  in  actual  situations  these  variables  may  not 
lend  themselvei  to  a:  simple  a presentation  as  is  made  in  the  drawing  below. 

The  line  marked  "transfer"  in  the  figure  is  a hypothetical  relation- 
ship between  degree  o " engineering  simulation  (of  one  or  more  variables  in 


mDC  TR  51;-563 


-21- 


Transfer  of  Training  Value 


Schematic  Relationship  boltween  Degree  of  Engineering 
Simulation,  Cost,  and  Transfer  of  Training  Value. 

(Assuming  motivation  tp  be  constant  and  high.) 
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the  operational  task)  and  the  transfer  value  which  may  arise.  At  very  low 
degrees  of  engineering  simulation  /Cl)  on  the  diagraiy  we  may  expect 
relatively  small  gains  in  transfer  value  with  given  increments  in  engineer- 
ing simulation.  This  statement  is  interpreted  as  follows:  With  low  degrees 
of  engineering  simulation  the  trainee  will  be  making  verbal  or  other 
mediations  of  both  display  and  control  components  of  the  task,  and  will  bo 
transferring  to  the  operational  task  little  more  than  the  identification 
of  controls  and  display  features,  nomenclature,  and  perhaps"  e kills  xn "gross 
procedures  limited  to  direction  of  movement  of  display  items  and  control 
items.1  He  may  also  be  acquiring  some  concepts  of  the  action  of  the  equip- 
ment in  some  context.  But  since  this  stage  of  learning  will  have  a high 
verbal  content  to  mediate  generalization,  increases  in  degree  of  engineer- 
ing simulation  will  tend  to  result  in  little  or  no  increase  in  transfer 
value  for  this  stage  of  learning. 

(2)  However,  as  learning  is  carried  to  higher  degrees,  especially  in 
tasks  requiring  perceptual-motor  coordinations  and  rapid  sequences  of 
accurate  perceptions  and  movements,  increments  in  engineering  simulation 
have  increasingly  higher  transfer  value,  assuming  the  training  on  the  train- 
ing device  is  carried  only  to  a degree  optimum  for  that  device.  But  degrees 
of  engineering  simulation  can  be  reached  where  dividends  by  way  of  transfer 
of  training  grow  less,  that  is,  diminishing  returns  set  in.  It  is  in  this 
region  that  production  costs  may  mount  disproportionately. 

(3)  Finally,  we  can  postulate  degrees  of  simulation  beyond  which  no 
appreciable  gains  in  transfer  can  be  found;  that  is,  where  human  vari- 
ability and  limitations  of  capacity  to  discriminate  situation  one  from 
situation  two  are  greater  than  variations  between  operational  and  training 
equipment,  or  are  greater  than  the  "reliability”  of  the  training  equipment 
during  its  operation. 

Unfortunately,  we  do  not  know  the  parameters  of  this  transfer  curve. 

Except  for  a few  scattered  reports  about  one  or  two  task  variables,  there  is 
available  little  information  from  which  an  actual  transfer  curve  could  be 
drawn.  Perhaps  one  of  the  principal  difficulties  has  been  that  the  various 
dimensions  of  the  problem  have  not  been  made  explicit.  An  attempt  will  be  made 
in  the  following  pages  to  do  so.  'When  the  various  premises  of  the  problem  have 
been  established,  it  is  to  be  hoped  that  some  conclusions  of  a practical,  even 
though  rule-of-thumb,  nature  can  be  drawn  until  more  experimental  evidence 
becomes  available. 


1.  Of  course,  if  the  above  is  all  that  the  training  task  need  teach,  then 
transfer  value  can  be  high  for  these  procedural  skills,  and  with  a low 
degree  of  engineering  simulation. 
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SIMULATION  AND  THE  SAMPLING  PROBLEM 

Lot  us  take  a different  tack.  Let  us  ask,  tfWhat  shall  ue  select  as 
our  operational  model  to  simulate  engineeringvise?"  What  mean  values  and 
tolerances  shall  be  used  in  the  construction  of  the  10  or  100  copies  of 
the  training  device?  Every  B-36  has  somewhat  different  characteristics  than 
every  other  B-36.  Which  B-36  should  we  copy? 

Sources  of  variability  will  be  discussed  under  two  main  divisions. 

One  of  these  divisions  will  discuss  the  variability  from  moment-to-moraent 
of  a given  unit  of  equipment.  The  other  sub-topic  will  be  the  variability 
between  different  units  or  samples  of  the  same  equipment. 

The  practical  outcome  to  the  manufacturer  of  this  discussion  on  vari- 
ability will  be  guides  as  to  tolerances  to  be  built  into  training  equipment. 


Within-Unit  Variation 

A given  aircraft  will  function  somewhat  differently  from  day  to  day. 

The  greater  the  extent  to  which  an  attempt  is  made  to  standardize  the  con- 
ditions of  its  operation,  the  less  variable  will  its  performance  be.  But  in 
practical  situations,  standardization  is  only  a matter  of  degree.  To  cite 
only  a few  sources  of  variability,  even  though  it  were  subjected  to  exactly 
the  sam9  situational  conditions,  there  are:  variations  in  maintenance,  age 
of  the  equipment,  fuel,  differential  types  of  wear  on  components  due  to 
special  history  of  the  equipment,  fuel  load,  loading  of  the  equipment,  stage 
of  warming  up,  and  so  on. 

So  we  can  expect  that  to  some  degree,  a person  operating  a given  piece 
of  equipment  will  have  a task  somewhat  different  each  time  he  operates  it. 

Insofar  as  the  operator  is  concerned,  these  physical  differences  may 
not  result  In  any  "practical"  difference  to  him  in  the  performance  of  his 
task  requirements.  Thus  he  may  not  be  able  to  report  any  difference  between 
equipment  N as  it  operates  today  and  as  it  operated  yesterday.  Nor  will  his 
task  necessarily  be  materially  different  in  performance.  Obviously,  there 
will  be  differences  whenever  the  error  components  of  a system  are  changed, 
but  these  differences  may  not  be  measurable,  or  they  may  not  fall  outside 
the  tolerance  criteria  established  for  effective  performance  of  th*  man- 
machine  task.  We  may  thus  speak  of  the  operator  as  being  insensitive  to  such 
machine  differences.  It  is  not  Important  at  this  point  whether  or  not  the 
operator  can  or  cannot  report  a difference:  what  is  important  is  whether  any 
difference  shows  up  in  his  task  performance  insofar  as  the  task  criteria  are 
concerned. 
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We  have  had  to  presume  that  tho  operator  himself  contributed  no  error 
to  the  man-machine  system.  This  assumption  is,  of  course,  quite  false. 

But  in  order  to  proceed  with  an  analysis  we  will  have  to  accept  certain 
"givens"  in  each  topic,  even  though  these  givens  become  variables  in  the 
next  topic. 

Sane  systems  of  equipment  are  more  subject  to  variability  than  are 
others,  although  of  course  variability  is  always  a relative  tern  and  it  is 
difficult  to  find  a conmon  denominator  on  which  to  base  direct,  quantitave 
comparisons. 

But  we  may  point  out  that  most  gunnery  systems,  for  example,  seem  to 
have  a good  deal  of  uncontrolled  machine  variation  under  operational  con- 
ditions. The  same  gun  will  change  in  its  barrel  temperature,  rifling,  and 
electrical  and  mechanical  linkages.  Differences  between  one  bullet  and 
another  will  show  up  in  hit  error.  So  if  we  could  substitute  for  the 
operator  an  aiming  device  which  was  itself  "perfectly"  reliable  we  would 
still  find  considerable  fluctuation  of  hit  error  from  one  shot  to  the  next, 
one  group  of  shots  to  the  next,  and  from  one  portion  of  the  gun's  history  to 
the  next.  We  have  been  presuming  that  we  are  sampling  from  a universe  of 
actual  operating  circumstances  such  as  occur  in  combat,  with  all  the  con- 
tributing circumstances  (such  as  ammunition,  maintenance,  target  movement) 
being  adequately  represented  in  our  samples. 

Radar  equipment  seems  also  to  suffer  even  yet  from  a fair  amount  of 
what  in  operatic  rehearsals  is  called  "temperament."  The  same  equipment 
functions  differently  at  different  times.  Obviously,  the  more  slipshod  the 
maintenance  is,  the  more  the  same  set  will  vary  from  one  mission  to  another. 
Presumably,  however,  we  aim  to  train  trainees  ultimately  to  cope  with  live 
operational  problems,  and  noo  merely  with  the  antiseptic  conditions  of  a 
laboratory. 

Let  us  refer  to  the  kind  of  variability  we  have  been  speaking  of  as  with' 
in-equipment  variation.  For  each  sample  from  a type  of  equipment,  there  will 
be  a mean  and  standard  deviation  of  performance  on  each  variable  of  its 
respective  outputs.  When  these  variables  interact,  their  individual  vari- 
ations may  become  compounded  into  still  greater  ranges  of  unpredictability, 
and  the  way  in  which  they  are  compounded  may  be  peculiar  to  each  individual 
sample  of  the  equipment  in  question. 

It  is  of  course  highly  time-consuming,  difficult,  and  expensive  to  de- 
termine within- equipment  variations , especially  if  this  determination  is  to 
be  done  unHer  operational  conditions  rather  than  an  artificial  laboratory- 
situation.  It  is  always  quite  safe  to  anticipate  (at  least  on  logical 
grounds)  that  the  variability  values  established  for  the  performance  of  a 
given  piece  of  equipment  will  be  much  larger  under  operational  use  than  in 
test  use.  The  total  error  variance  (a  measure  of  variability)  of  a complex 
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man-machine  system  will  increase  according  to  the  well  known  sum  of  squares 
principle. 1 

Unless  actual  determination  of  within-equipment  variability  is  determined 
under  suitably  representative  operational  conditions,  we  cannot  know  the  lower 
limits  of  the  precision  needed  to  construct  a training  device  for  this  equip- 
ment. Consequently,  we  will  not  know  how  cheaply  we  can  afford  to  build  it. 

Furthermore,  by  building  a training  device  which  performs  in  a more 
standard  manner  than  a given  piece  of  equipment  is  likely  to  perform,  we  risk 
a biased  training  of  the  trainee  by  shielding  him  from  ranges  of  machine 
variation  which  contribute  to  the  performance  of  the  task  in  operations.  Ad- 
justing to  such  variations  may  be  an  important  part  of  training. 

Between-Unit  Variation 


Even  when  two  units  of  the  same  operational  equipment  cone  off  the  as- 
sembly line  together,  they  will  vary  to  some  degree  in  their  functional  proper- 
ties. The  more  complex  the  equipment  and  the  more  complex  the  situation  and 
variables  in  which  they  will  operate,  tho  greater  these  differences  between 
individual  equipments.  They  will  differ  according  to  mean  values  of  perform- 
ance and  also  according  to  the  variation  each  shows  around  its  mean  perform- 
ance when  subjected  to  comparable  conditions.  Thus  aircraft  Helen  of  type  XD 
will  stall  at  an  airspeed  of  70  mph,  with  a standard  deviation  of  li  mph, 
while  aircraft  Josephine  of  type  XD  will  stall  at  66  mph,  with  S.D.  of  3 mph. 

A third  example,  (Dorothy)  will  stall  at  a different  speed  with  a different 
standard  deviation. 

Our  problem  is,  since  we  want  to  train  a pilot  to  fly  aircraft  type  XD, 
shall  we  train  him  on  a device  simulating  Helen,  Josephine,  or  Dorothy? 

(In  the  particular  example  we  have  chosen,  these  differences  in  stalling  speed 
may  be  psychologically  unimportant.  The  example  is  merely  for  illustrative 
purposes.) 

The  same  sources  which  make  for  variability  from  time  to  time  within  an 
individual  equipment  will  be  acting  to  widen  the  possible  range  of  differences 
between  individuals  of  an  equipment.  The  older  that  an  individual  equipment 
becomes,  the  more  likely  it  will  develop  individual  characteristics  as  a 
function  of  its  intrinsic  difference  in  construction,  and  also  as  a function 
of  its  individual  history  or  experience. 


1.  The  total  variance  is  equal  to  the  sum  of  the  independent,  contributory 
sources  of  variance.  If  these  sources  are  not  independent,  the  total  vari- 
ance is  equal  to  the  sum  of  the  variances  from  each  source  plus  the  cor- 
relational factors.  (See  any  standard  statistics  text.) 
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The  presentation  of  the  problem  of  within-and  between-unit  varia- 
tion poses  significant  logical  problems  to  the  training  device  engineers 
even  before  He  come  to  the  psychological  problems.  The  psychological  pro- 
blems consist  of*.  (1)  what  are  the  equipment  differences  which  make  a 
psychological  difference,  and  (2)  how  important  is  this  psychological  dif- 
ference to  training  efficiency  and  transfer  of  training.  A related  problem 
is,  how  far  can  training  go  on  a so-called  simulator,  and  at  what  level  of 
skill  or  training  is  it  more  profitable  to  shift  the  trainee  to  the  opera- 
tional equipment,  taking  into  account  our  logical  problems  in  sampling 
equipment  characteristics  as  well  as  other  difficulties  in  the  training 
situation?  We  may  find  that  the  human  being  is  so  insensitive  cr  so  adapt- 
able, that  what  constitute  frightening  variations  in  the  manufacturing  process- 
es of  equipment  are  insignificant  in  the  realistic  training  context. 

Conclusions  To  The  Sampling  Problem  In  Physical  Simulation 

If  one  and  only  one  sample  of  the  operational  equipment  is  used  as  a basis 
for  determining  the  engineering  characteristics  of  a training  device,  the  par- 
ticular one  unit  selected  as  the  model  may  not  be  "representative”  (have  aver- 
age performance  value)  of  all  the  other  units  or  items  of  that  class  of  equip- 
ment. Thus  the  particular  F-500  chosen  for  data-gathering  purposes  may  be 
consistently  more  sluggish,  or  more  sensitive,  than  nearly  all  the  other  r-500>s 
which  will  be  used  in  operations.  A device  which  physically  simulates  this 
F-500  will  not  simulate  other  F-500‘s.  Thus  we  risk  biased  training. 

Furthermore,  by  building  a training  device  more  standardized  than  ia  opera- 
tional equipment,  we  risk  a different  kind  of  biased  training  by  shielding  the 
trainee  from  ranges  of  machine  variation  which  contribute  to  the  task  in  opera- 
tions. Adjusting  to  such  variations  may  be  an  important  part  of  training.  In 
actual  fact,  it  is.  Pilots  do  get  out  of  one  F-8h  and  without  special  "train- 
ing" fly  off  in  another  F-8U  which  has  a somewhat  different  feel  and  behavior 
characteristics. 

So  we  see  that  even  if  we  accept  physical  simulation  as  a legitimate  ap- 
proach to  providing  a behavioral  context  for  training  or  performance,  we  are 
balked  by  problems  which  seem  to  have  no  answer  in  a reasonable  economy. 

It  is  true  that  many  of  the  problems  cited  above  are  characteristically 
. aored  by  basing  the  specifications  of  the  training  device  not  on  the  actual, 
measured  operating  characteristics  of  the  operational  equipment.  The  train- 
ing device  is  built  to  the  calculated  specif ioatlona  of  the  operational  equip- 
ment. The  matching  of  these  specifications  to  what  the  operating  equipment 
actually  does  rarely  is  more  than  piecemeal,  and  confined  to  one  or  two  accept- 
ance and  engineering  test  models.  These  test  models  are  not  necessarily  simil- 
ar to  the  production  models  of  the  operating  equipment.  Undoubtedly  they  are 
better  maintained. 
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Designed  by  this  procedure,  the  training  device  simulates  not  an  actual 
operating  device  but  the  specifications  for  an  operating  device.  Dis- 
crepancies may  be  far  more  substantial  than  the  close  manufacturing  toler- 
ances usually  imposed  on  the  construction  of  simulators. 


Sampling  Of  Program  Data 

Programs  consist  of  environmental  circumstances  with  which  the  man- 
machine  team  will  have  to  cope.  We  may  think  of  the  characteristics  of  the 
machine  itself  as  the  properties  of  the  diBplay-control  or  cursor-control 
system;  and  the  characteristics  of  the  programs  as  target  plus  noise  proper- 
ties in  the  learning  problem.  Obviously  there  is  always  interaction  between 
the  cursor-control  system  and  programs.  That  is,  no  man-machine  operation  is 
performed  in  an  environmental  vacuum. 

In  gunnery,  whether  fixed  or  flexible,  the  target  may  vary  in  identity, 
size,  and  capacity  for  evasive  maneuver.  Ccmbat  may  occur  at  high  or  low 
altitudes,  against  the  sun  or  away  from  the  sun,  among  clouds  or  in  clear 
skies;  if  radar  is  used,  countermeasures  may  be  employed  or  the  set  may  mal- 
function in  part  or  completely.  Each  man-machine  system  has  a variety  of 
program  variables,  and  these  variables  need  to  be  sampled  systematically  be- 
fore training  can  be  said  to  be  complete  for  operational  readiness. 

The  distinction  between  engineering  simulation  and  psychological  simu- 
lation applies  to  program  variables  as  well  as  to  the  physical  characteristics 
of  the  equipment  in  the  man-machine  system.  In  training  it  should  be  the  aim 
to  provide  those  cues  sufficient  and  necessary  for  learning  to  perform  the 
operational  task.  The3e  cues  may  in  many  cases  be  less  than  complete  realism. 

Physical  realism  is  the  relatively  safe  way  to  include  the  sufficient  and 
necessary  along  with  the  irrelevant  in  cues.  But  it  is  frequently  so  expensive 
that  few  can  be  built  and  maintained,  so  the  large  amount  of  time  which  the 
many  trainees  may  require  to  attain  proficiency  is  impossible  to  provide.  Thus 
there  is  lost  a principal  aim  of  the  training  device  which  is  to  provide  ample 
practice  time  for  trainees  to  become  highly  skilled. 

Furthermore  physical  realism  will  not  guarantee  that  all  important  program 
variables  will  be  represented,  or  represented  in  sufficient  range,  or  in  suf- 
ficient interactions  with  each  other.  In  visual  reconnaissance,  one  class  of 
programs  would  deal  with  reduced  visibility  due  to  haze,  glare  (atmospheric 
conditions),  others  would  consist  of  target  size,  target  camouflage,  variation 
in  aircraft  altitude  and  attitude,  perceptions,  of  emergency  conditions  and 
many  others.  A visual  reconnaissance  pilot  trained  to  observe  from  one  altitude 
would  lack  important  skills  for  reconnoitering  at  different  altitudes.  The  com- 
bined conditions  of  camouflaged  targets  and  variable  altitudes  provide  an 
example  of  interaction  of  variables  in  a program. 
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In  order  laat  the  usefulness  of  the  trainer  not  be  severely  restricted, 
there  must  be  capacity  for  providing  programs  adequate  in  kind  and  range. 

But  the  trainoe  may  learn  to  make  the  correct  responses  to  a given 
practice  program  after  relatively  few  trials.  In  order  to  train  him  to  cope 
with  tiugtkind  of  program,  a greater  number  of  samples  are  necessary  than  he 
can  memorize  as  such,  unless  operations  demand  only  a restricted  set  of 
habits  as  for  example  "following  of  standardized  procedures." 

Before  undertaking  to  design  a device  capable  of  incorporating  all  uro- 
grams and  tanks  encountered  in  operations,  the  entire  training  requirement 
should  be  examined  with  a view  to  what  tasks  and  programs  can  be  given  train- 
ing in  part- task  trainers,  procedural  trainers,  part-simulators  and  the 
operational  equipment.  In  this  way  costs  can  become  reasonable  and  training 
devices  can  be  produced  in  quantity  and  maintained.  It  is  no  criticism  that 
a synthetic  trainer  does  not  in  itBelf  train  completely  all  task  requirements 
to  operational  readiness— unless  the  claim  of  completeness  is  made  for  it. 


StMMART  AND  SUGGESTIONS 

It  has  been  established  that  physical  simulation  is  a matter  of  kind  and 
degi’ee.  Problems  have  been  raised  as  to  the  model  or  sample  to  be  used  in  de- 
termining the  operating  characteristics  and  programs  in  the  design  of  the  train- 
ing device.  Economical  solutions  to  these  problems  of  physical  simulation  do 
not  seem  likely.  What  is  the  alternative? 

The  alternative  is  to  ask,  "With  respect  to  a given  purpose  such  as  train- 
ing and  transfer  of  training,  how  much  physical  difference  makes  little  or  no 
psychological  difference?"  ¥e  may  find  that  very  considerable  physical  differ- 
ences are  negligible  in  transfer  of  training.  And  this  may  be  especially  true 
of  some  tasks  more  than  others,  and  some  stages  of  training  more  than  others. 
Furthermore,  we  may  be  able  to  train  around  limitations  in  physical  replica- 
tion. Finally,  we  may  be  able  to  see  the  training  device  and  synthetic  train- 
ing in  a perspective  which  i3  boxed  in  less  by  the  state  of  the  engineering 
art  than  by  our  knowledge  of  human  behavior. 

A number  of  recommendations  can  be  drawn  from  this  chapter.  They  are  as 
f ollovjs : 

1.  The  design  of  training  devices  should  be  directed  towards  maximum 
transfer  of  training  value,  not  physical  realism. 

2.  Physical  realism,  both  in  degree  and  kind,  finally  reaches  diminish- 
ing returns  in  transfer  of  training  value  per  developmental  dollar. 
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3.  Some  stages  of  training  and  kinds  of  task  trained  require  less 
physical  realism  than  others.  Account  should  be  taken  of  thoso  differences. 

Tolerances  of  a variable  in  the  trainer  should  take  into  account, 
and  generally  not  be  required  to  be  closer,  than  the  general  range  of  varia- 
tion found  in  that  variable  in  tiems  of  operational  equipment.  Tolerances 
required  in  training  equipment  should  also  take  into  account  that  the 
characteristics  of  the  same  item  of  operational  equipment  vary  considerably 
from  time  to  time. 

5.  The  vital  importance  of  providing  practice  programs  of  sufficient 
kind,  range  and  variety,  justifies  concentrated  attention  of  the  designer. 

The  trainee’s  range  of  skill  will  tend  to  be  restricted  to  the  range  and 
complexity  of  problems  he  has  met  in  training. 

6.  The  kinds  and  extent  of  physical  realism  built  into  a given  training 
device  should  be  based  upon  careful  examination  of  what  is  psychologically 
important  versus  what  is  unimportant. 


The  next  chapter  is  an  approach  to  the  design  of  training  devices  from 
the  direction  of  psychological  requirements  for  transfer  of  training  rather 
than  physical  simulation. 
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STAGE  OF  LEARNING  AND  DEGREE  OF  PHYSICAL  SIMULATION 


Ono  does  not  begin  learning  a complex  job  or  group  of  tasks  by  starting 
to  learn  everything  at  once.  There  are  discontinuities  in  what  is  learned 
and  how  it  is  learned.  These  discontinuities  occur  spontaneously  and  they 
may  show  up  more  in  some  individuals  than  in  others. 

In  other  words,  the  human  learning  process  consists  of  a group  of  pro- 
cesses, some  going  on  at  the  same  time,  others  evolving  from  response 
groundwork  acquired  through  practice  on  the  task. 

These  stages  or  phases  of  learning  may  determine  the  kinds  and  degrees 
of  physical  simulation  sufficient  for  effective  training  and  transfer  of 
training  both  to  later  training  and  to  operational  equipment. 

The  following  pages  of  this  chapter  will  attempt  to  rolate  phase  of 
learning  with  minimum  simulation  or  other  training  device  requirements.  It 
will  be  seen  that  different  stages  of  learning  will  call  for  different  mini- 
mum trainer  requirements.  But  these  recommendations  should  not  imply  that 
a different  training  device  should  therefore  be  built  for  each  stage  dis- 
cussed, nor  that  a training  device  may  not  be  satisfactory  for  training  if 
it  incorporates  more  than  the  minimum  as  proposed. 

Furthermore,  in  actual  learning,  many  of  these  phases  will  overlap  each 
other,  or  shade  imperceptibly  one  into  another.  Therefore  it  would  be  false 
to  consider  the  process  described  in  each  phase  as  independent  of  other 
phases  or  aspects  of  ongoing  learning  processes. 


EARLY  STAGES  IN  TASK  LEARNING 

The  learning  of  practically  every  task,  whether  continuous  or  discon- 
tinuous, begins  by  getting  ideas  on  what  the  task  is  about,  and  how  to  go 
about  it.  Evan  continuous  tracking  ta3ks  have  at  least  brief  phases  during 
which  they  are  performed  like  discontinuous  procedures. 

can  roughly  describe  Stage  One  as  the  learning  of  procedures,  but 
not  learned  by  any  degree  where  the  task  responses  are  unreeled  automati- 
cally. The  trainee  is  learning  a repertory  of  self  instructions,  concepts, 
and  images  of  what  he  is  supposed  to  do  and  under  what  conditions.  Much  of 
this  learning  cm  be  divided  between  classroom  study,  texts,  training  aids, 
and  trainers  whi6h  are  extremely  rough  or  non-function&l  copies  of  opera- 
tional eqcipeiont.  iSTthese  latter  the  trainee  wilT get  practice  in  grossly 
carrying  out*  instructions  provided  by  himself  or  supplemented  ty  the  instruct- 
or or  a test.  During  this  stage  of  learning,  the  instructor  will  usually  be 
the  most  economical  source  cf  knowledge  &f  results.. 
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Stage  of  learning  and  simulation 
Early  learning 


Learning  What  The  Equipment  Is  Supposed  To  Do 

Tlvia  part  of  early  learning  is  one  phase  of  what  may  be  called  indoct- 
rinational  training.  Not  only  should  it  cover  the  principal  purposes  and 
objectives  which  the  operational  man-machine  teams  are  assigned  to  achieve, 
but  the  degree  of  precision  of  various  aspects  of  their  criterion  performance 
may  be  presented.  How  the  particular  tean^s  work  fits  in  with  other  jobs  in 
the  performance  of  a mission  may  also  be  presented.  For  example,  the  relation- 
ship of  the  particular  fighter  aircraft  to  the  bomber  for  which  it  flies  as 
protection  would  be  described. 

Unless  this  kind  of  information  is  made  explicit  to  the  trainee,  he  will 
develop  his  own  hypotheses  which  may  or  may  not  fit  the  facts. 


TRAINER  RECOMMENDATIONS  FOR  TEACHING  WHAT  THE  EQUIPMENT  IS  SUPPOSED  TO  DO 

1.  If  possible,  experienced  operators  should  demonstrate  criterion 
performance  on  the  OPERATIONAL  EQUIPMENT. 

2.  TRAINING  FILM  may  be  used  to  demonstrates 

a.  The  environmental  contexts  of  operations. 

b.  The  integration  of  part-task  outcomes  with  other  part-task  out- 
comes; or  how  the  output  of  the  skilled  operator  realistically 
affects  or  interacts  with  some  greater  system,  mission  or  enter- 
prise. (For  example,  how  the  interceptor  pilot  interacts  with 
ground  control  operations  and  with  the  entire  defense  system). 
Actual  scenes  of  action  should  be  integrated  with  animated  dia- 
grams which  schematize  the  relationships. 

c.  How  one  phase  of  training  fits  in  with  later  phases  of  training. 
The  film  shoulH"” include  action  scenes  of  actual  trainee  perform- 
ance at  such  stages;  the  action  should  include  and  point  out 
characteristic  error  and  degree  of  error  in  task  or  job  perform- 
ance. Later  sequences  may  show  the  improvement  in  actual  job 
outcome. 

3.  DEMONSTRATION  EQUIPMENT:  Where  possible,  simple  graphic  aids  and 
small-scale  models  may  be  used.  Autssnatic  operation  of  moving  parts 
in  demonstrators  probably  does  not  have  much  payoff  value  as  against 
instructor  controlled  demonstrators. 

u.  TRAINING  DEVICES  SUCH  AS  "SIMULATORS"  may  be  demonstrated  by  well- 
motivated  individuals  who  do  not  perform  casually  or  as  if  error 
could  not  be  committed  on  thc""acvics. 

Learning  what  the  equipment  is  supposed  to  do  and  the  role 
of  the  human  operator  in  its  performance  will  have  primarily 
motivational!  value  to  the  trainee.  The  indoctrination  phase  will 
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tend  to  color  the  trainee^  attitudes  and  motivations 
towards  training,  training  devices  and  training  pro- 
grams. The  reconnendations  proposed  here  recognize  that 
group  morale  and  individual  effort  rest  on  having  clear- 
ly defined  goals  and  routes  and  stages  for  reaching  the 


Before  the  trainee  can  carry  out  verbal  instructions  about  equipment 
(including  instructions  he  Might  give  to  hl»self)  he  must  learn  to  pair  the 
names  of  objects  with  the  work  objects  themselves.  Thus  he  learns  that  video 
gain  knob,  altimeter,  trim  tab  control,  range  reticle,  are  names  for  objects 
he  can  perceive  in  his  work  space.  Trainers  for  teaching  nomenclature  might 
be  no  more  than  charts  or  blackboard  sketches.  If  location  and  simple  on-off 
switching  procedures  are  also  to  be  taught,  a simplified  mockup  made  of  inert 
materials  may  be  used.  This  type  of  equipment  iB  suitable  for  maintenance 
men  who  use  automatic  checking  devices  and  make  repairs  on  a black  box  re- 
placement level.  There  will  also  be  nomenclature  associated  with  work  process- 
es and  work  products. 


TRAINER  RECCHHEUMTIONS  FOR  TEACHING  NCHMCIATURE  OF  THE  EQUIPMENT 


1.  Learning  of  nomenclature  can  be  begun  with  PICTURES  and  DIAGRAMS,  and 
continued  with  NON-FUNCTIONAL  MOCK-UPS  of  the  operational  equipment. 
It  would  be  uneconomical  to  tie  up  operational  equipment  or  expensive 
trainers  for  training  on  vocabulary. 


Operational  equipment;  If,  however,  operational  equipment  is  normal- 
ly idle  and  available,  it  is  desirable  and  economical  to  use  it  for 
training  in  both  nomenclature  and  locations  of  objects.  Labelled 
diagrams  or  pictures  should  be  accessories  provided  the  trainee. 


Number  of  trainers  required;  Sufficient  numbers  of  copies  of  the 
"nomenclature  trainers"  should  be  available  to  permit  each  trainee 
considerable  practice  for  learning  "naming"  habits. 

Without  an  extremely  high  degree  of  skill  in  matching  the  object 
with  the  word  or  symbol,  the  traine  will  be  confused  in  following 
instructions  and  in  understanding  and  applying  criticisms  in  training. 


Locations  of  Objects  In 


i@nt  And  In 


The  trainee  must  also  learn  the  location  of  the  work  objects  he  can  iden- 
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tify  and  name.  The  learning  of  locations  ant'  ths  learning  of  nomenclature  is 
probably  best  done  in  the  same  training  oxerr  aes. 

The  extent  to  which  the  trainoc  can  rapidly  follow  cowiands  in  locating 
and  identifying  a display  or  control  object  will  parallel  the  extent  to  which 
ho  can  follow  ins  unctions  without  search  and  confusion.  It  will  also  be  the 
extent  to  which  he  can  readily  imagine  symbolic  activities  when  he  is  given 
instructions  or  information  apart  from  his  ma’  ing  an  overt  response. 

Much  valuable  time  in  an  advanced  trainer  can  be  saved  if  the  trainee 
brings  to  it  well  integrated  habits  of  locating  the  objects  in  his  work  en- 
vironment. 


TRAINER  RECOMMENDATIONS  FOR  TEACHING  LOCATIONS  OF  OBJECTS 


1.  PICTURES,  plus  NON-FUNCTIONAL  MOCK-UPS  will  be  adequate  for  training 
in  locations  of  objects. 

2.  Immediate  knowledge  of  results;  Learning  rate  may  be  increased  by 
providing  for  immediate  knowledge  o'  results.  Thu*,  adjacent  to  the 
picture  or  mock-up  there  may  be  a l^-t  of  the  names  of  objects  to  be 
located.  Each  item  in  the  list  ha  a light  and  a push-button  switch 
next  to  it,  and  each  matching  item  the  picture  or  mock-up  has  a 
push  button  and  switch.  When  the  trainee  is  learning  nomenclature 
and  locations,  in  the  absence  of  the  instructor,  he  may  read  the 
name  of  an  item,  press  the  adjacent  switch  and  note  the  object  be- 
side which  a light  glows.  Conversely,  he  may  press  the  switch  be- 
side the  depiction  of  the  object  and  note  beside  what  name  the  light 
glows.  He  may  aL'o  use  the  device  for  testing  himself. 

Such  a device  would  be  especially  helpful  where  large  arrays  of 
items  must  be  quickly  learned  with  r spect  to  nomenclature  and  loca- 
tions. 


3.  Breaking  large  numbers  of  items  into  smaller  groups;  Large  groups  of 
paired  items  can  lie  best  memorized  by  breaking  them  into  smaller 
groups  ox  items.  If  a group  0:  it^s  can  be  related  to  each  other 
logically  (as  by  performing  some  common  or  related  function),  the 
training  aid  Bhould  group  items  accordingly.  The  grouping  should,  of 
course,  be  coded  by  color,  or  dotted  bands,  rather  than  by  changes  in 
normal  spatial  configuration. 

k»  Coding  of  items  according  to  se coerce  of  uses  If  a group  of  displays 
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and  controls  figure  in  a series  of  job  responses,  the  individual 
items  may  be  artificially  number  coded.  For  example,  a mechanic 
may  have  a standard  check  routine.  If  the  trainee  learns  names 
and  locations  of  objects  in  the  numerical  sequence  of  later  use, 
he  will  be  learning  more  per  unit  of  practice  than  by  learning 
items  first  in  random  order  and  later  in  sequence. 

Search  And  Scanning  Hethdda 

It  is  likely  that  the  search  and  scan  methods  adopted  and  practiced  by 
tic  trainee  early  in  learning  will  tend  to  persist  unless  actively  unlearned 
and  supplanted  by  substitute  habits.  In  many  military  and  industrial  situ- 
ations total  operational  effectiveness  requires  a wide  perceptual  scope.  Un- 
der conditions  of  stress,  and  similarly  under  conditions  of  highly  repetitive 
practice  of  routine  activities,  perception  tends  to  become  restricted  to  what 
have  been  called  "strip-maps'1  of  the  work  environment. 

Thus  the  gunner  while  tracking  a target  should  scan  lest  he  become  a tar- 
get to  a third  aircraft;  bombardier  and  navigator  should  be  alert  to  gross  dis- 
crepancies or  mis identifications  of  terrain;  the  pilot  snould  be  aware  of  other 
aircraft  in  his  neighborhood;  all  operators  should  become  quickly  aware  of 
equipment  malfunctions.  But  these  outcomes  presuppose  adequate  scanning  and 
perceptual  search  techniques  which  will  not  tend  to  arise  spontaneously,  nor 
pan  they  readily  be  overlaid  on  other  habit  systems. 

Even  part-trainers  should  provide  for  keeping  the  trainee  alert  to  the 
broader  aspects  of  his  work-space  than  may  be  required  by  part-task  perfor- 
mance. This  provision  should  almost  certainly  be  included  before  the  part-task 
is  learned  to  a high  degree,  but  not  before  tie  trainee  has  mastered  the  locations 
of  objects  in  his  work  configuration. 


TRAINER  RECCMM0JDATIONS  FOR  SCANNING  METHODS 

1 6 Stimuli  occasionally  presented  outside  the  primary  work  area: 

Training  for  scanning  habits  may  be  introduced  by  peripheral  signals 
to  which  the  trainee  need  respond  only  by  recognition  of  the  presence 
of  the  peripheral  stimulus.  Consequently  the  engineering  simulation 
may  be  rough  and  even  merely  symbolic;  thus  in  synthetic  gunnery 
training  the  presence  of  another  enemy  aircraft  may  be  denoted  by  the 
lighting  of  weak  intensity  wheat  grain  lights  to  which  the  trainee 
responds  by  throwing  some  arbitrary  switch.  The  distributor  of  such 
peripheral  cues  should  sample  from  the  work-space  in  which  critical 
cues  may  appear.  These  cues  should  be  about  as  difficult  to  see  (at 
least  in  later  training  stages)  as  the  operational  objects. 

2.  Signals  within  the  primary  work  area:  If  a trainee  is  engaged  in 
practice  on  a part-task,  he  also  should  be  required  occasionally  to 
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note  changes  in  displays  within  the  work  area  which  are  not  relevant 
to  the  task  on  which  he  is  concentrating  practice.  Thus  if  a trainee 
is  learning  to  fly  straight  and  level,  there  nay  be  occasional  gross 
changes  in  the  engine  temperature  indicator,  or  the  fuel  indicator. 
These  display  changes  may  be  gross,  and  even  all-or-none,  their  occur- 
rence should  be  under  the  control  of  the  instructor.  Response  adequacy 
of  the  trainee  who  must  detect  these  conditions  may  be  Judged  by  the 
instructor. 


Learning  What  Control  Affects  What  Display 

Another  preliminary  phase  of  learning  a new  job  is  discovering  and  memor- 
izing what  controls  affect  what  display  features.  The  term  "display0  should 
include  both  instrumental  and  environmental  sources  of  cues. 

Thus,  fore-and-aft  movement  of  the  joystick  results  in  a shift  in  the 
pointing  of  the  nose  of  the  aircraft  (up  to  a point),  or  a change  in  the  read- 
ings on  an  artificial  horizon  and  perhaps  an  altimeter. 

Separate  training  devices  would  not  ordinarily  be  necessary  for  this  phase 
of  learning.  It  is  possible,  however,  that  if  a very  complex  control  board  of 
switches  was  a part  of  the  operational  equipment,  some  pre-training  in  the  iden- 
tity of  control-display  interacters  would  reduce  need  for  time  on  more  complex 
trainers. 


TRAINER  REGCHMENDATIONS  FOR  LEARNING  CONTROL-DISPLAY  INTERACTORS 

1.  DIAGRAMMATIC  PHOTOGRAPHS  may  be  used  to  show  connections  between  con- 
trol items  and  display  items  and  provide  pre-training. 

2.  MOCK-UPS  OR  PANEL  DISPIAYS  with  simple  mechanical  linkages  between  con- 
trol and  display  will  be  sufficient  to  teach  verbal  understanding  of 
what  control  affects  what  display.  The  mock-up  may  provide  actual 
practice. 

Usually,  this  training  will  actually  be  include:’  in  the  phase 
next  described.  So,  from  a practical  standpoint,  learning  what 
control  affects  what  display  and  control-display  directional  re- 
lationships may  be  treated  as  a single  training  problem. 

Control-Display  Directional  Relationships 

In  this  aspect  of  learning,  the  trainee  discovers  and  memorizes  the  dir- 
ection a display  changes  when  its  matched  control  is  moved.  Pushing  the  accel- 
erator forward  increases  a speedometer  reading  (or  rate  at  which  the  roadway 
flows  past  the  field  of  view) ; whereas  pushing  the  brake  pedal  has  the  reverse 
affect. 
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Where  only  one  variable  is  involved,  or  no  wore  than  ono  or  two  controls 
are  within  the  work  environment,  and  where  feedback  dolay  intervals  are  very 
short,  this  phase  of  learning  will,  of  course,  be  very  rapid.  But  where- this 
is  not  the  case,  as  for  example  in  the  multiple  adjustments  which  a mainten- 
ance mechanic  on  electronics  equipment  has  to  make,  learning  nay  require  con- 
siderable practice. 

If  the  trainee  has  previously  performed  on  equipment  which  has  reverse 
control-display  relationships,  extensive  practice  may  be  required  under  a 
variety  of  programs,  including  time  stress,  before  the  old  habits  get  reli- 
ably eliminated.  Even  the  disappearance  of  tho  old,  but  now  erroneous,  habit 
from  one  training  session  is  no  guarantee  of  its  final  elimination.  It  may 
recur,  though  at  lesser  strength,  in  later  training  sessions  or  operations. 


TRAINER  RECOMMENDATIONS  FOR  CONTROL-DISPLAY  DIRECTIONAL  RELATIONSHIPS 

1.  Functional  mock-ups  will  be  adequate,  and  physical  simulation  (in- 
cluding lag)  may  be  extremely  rough. 

2.  Practice  exercises  could  present  discrete  misalignments  of  displays 
which  'could  be  zeroed  by  moving  the  appropriate  control  in  the  pro- 
per direction.  Dynamic  characteristics  need  not  be  reproduced. 

3.  Reduction  of  feedback  lag;  If  in  operations  there  are  lags  of  more 
than  a few  seconds  between  control  action  and  feedback  display, 
these  and  even  lesser  lags  could  be  eliminated  in  the  trAdner  intend- 
ed only  for  practice  in  these  early  stages  of  learning. 


NOTE;  A number  of  training  aids  and  devices  have  been  recom- 
mended thus  far.  In  each  case,  the  minimum  requirement  by  way 
of  construction  has  been  indicated.  It  is  likely,  however, 
that  the  serai-functional  mock-up  described  immediately  above 
would  be  the  best  bet,  plus  a set  of  labelled  photographs,  to 
enable  the  trainee  to  get  not  only  familiarity  with  a complex 
job  through  his  eyes  and  muscles,  but  also  to  get  a fairly 
clear  idea  of  what  he  is  supposed  to  do  and  when.  Call  it,  or- 
ientation, or  pre-training,  or  learning  the  elementary  proce- 
dures—or  what  you  will— this  type  of  practice,  if  properly 
monitor sd  or  supervised,  can  reduce  much  of  the  trainee's  later 
confusion.  This  confusion  may  arise  when  he  must  not  only  know 
the  general  pattern  of  doing  a task,  but  must  also  properly 
grade  and  time  his  responses,  and  perhaps  also  make  complex  de- 
cisions. Because  it  is  difficult  to  "think  of  more  than  one 
thing  at  a time"  it  is  necessary  to  learn  by  organizing  complex 
response  items  into  patterns.  The  devices  thus  far  described 
can  be  helpful  in  establishing  such  patterns  by  providing  the 
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trainee  with  the  proper  exercises.  Thus  the  more  complex  train- 
ers and  simulators  can  be  freed  to  teach  levels  of  skill  which 
the  above  devices  cannot  do,  but  which  usually  demand  more  prac- 
tice time  than  trainees  are  able  to  get  if  complex  trainers  must 
be  used  for  11  pre- training 11  as  well  as  training. 

Elementary  Control-Display  Ratio 

If  he  is  given  the  opportunity,  the  trainee  will  begin  to  learn  tho 
amount  of  control  movement  which  results  in  a given  amount  of  display  move- 
ment or  change.  Accuracy  will,  of  course,  continue  to  increase  until  it 
levels  off  towards  a ceiling  at  high  degrees  of  skill.  With  suitable  prac- 
tice, responses  will  show  progressively  less  terminal  adjustment  and  oscilla- 
tion especially  where  a cursor  position  must  be  realigned  to  some  changed, 
steady  state  as  indicated  on  a display. 

The  entire  matter  of  simulation  of  control-display  ratio  is  one  of  the 
most  important  in  training.  It  is  risky  to  make  inferences  in  this  area,  and 
research  is  scanty  and  far  from  systematic  with  respect  to  type  of  task,  degree 
of  learning,  control-display  lag  and  other  factors  which  probably  interact  with 
fidelity  of  control-ratio  in  transfer  of  training. 

But  where  discrete  control  movements  produce  fairly  discrete  display  con- 
sequences, especially  without  time  lags  beyond  a second  or  two,  it  is  probable 
that  only  gross  fidelity  in  control-display  ratio  is  necessary  to  maintain  op- 
timal training  value  in  early  stages  of  learning  the  task  or  job.  We  may  con- 
sider this  early  learning  as  almost  entirely  a learning  of  procedures,  even 
where  continuous  or  tracking  tasks  are  involved,  and  especially  if  the  trainee 
is  learning  to  coordinate  a number  of  stimulus -response  patterns  at  about  the 
same  time. 

Statements  on  simulation  of  control-display  ratio  for  middle  and  higher 
stages  of  learning  are  necessarily  complex.  These  statements  are  presented  in 
later  paragraphs  and  sections  of  this  chapter,  and  another  general  discussion 
of  simulation  of  control-display  ratio  is  provided  in  Chapter  II. 


TRAINER  RECOMMENDATION  FOR  ELEMENTARY  CONTROL- DISMAY  RATIO 

No  general  recommendation  can  be  offered.  It  seems  unlikely  that  an. 
"intermediate”  training  device  with  only  coarsely  approximate  control-display 
ratios  would  be  very  economical  in  a total  training  picture.  It  is  possible 
that  "part-task"  trainers  can  be  developed  which  would  require  only  coarse 
control-display  ratio  and  rate  relationships.  For  example,  the  teaching  of 
aerial  reconnaissance  to  experienced  pilots  in  a synthetic  device  might  re- 
quire coarse  aircraft  control=display  ratios:  presuming  that  the  flying  of 
the  aircraft  would  here  simulate  the  "division  of  attention"  requirement. 
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Control-Display  Ranges 

The  range  effect  is  3hovn  by  the  tendency  for  human  judgments  to  gravi- 
tate towards  the  average  of  their  experiences  and  to  be  modified  by  the  fre- 
quency and  range  of  values  experienced  in  some  variablo.  Thus  the  automobile 
driver  who  has  never  driven  above  hO  moh  may  be  required  in  an  emergency  to 
speed  up  to  60  mph  in  order  to  get  through  a tight  3pot.  He  must  judgo  what 
speed  is  called  for.  However,  he  will  tend  to  feel  that  an  actual  60  mph  is 
too  fast  even  for  this  emergency.  A hypothetical  driver  who  "always"  drove 
between  70  and  100  mph  would  tend  to  judge  that  60  mph  was  too  slow  for  the 
objective  situation  vhich  actually  -''lied  for  60  mph. 

In  order  to  avoid  false  range  effects  which  may  creep  into  perceptual 
"habits/1  the  trainee  should  probably  be  given  early  in  his  training,  some 
experiences  with,  the  full  dynamic  ranges  of  the  capacity  of  the  operational 
equipment  to  respond.  The  actual  limits  of  these  ranges  may  be  fairly  coarse- 
ly copied,  especially  since  there  is  likely  to  be  fairly  large  between-unit 
variation  in  operational  equipment  (see  Within-Unit  Variability  in  Chapter  II), 

Since  range  effects  seem  to  develop  quite  rapidly  and  we  do  not  know  how 
persistently  they  may  effect  new  experiences,  it  is  suggested  that  they  be  con- 
sidered quite  early  in  training. 


TRAINER  RECOMMENDATIONS  FOR  CONTROL-DISPLAY  RANGES 

1,  Programs  or  exercises  in  training  should  occasionally  require  the 
trainee  to  test  the  "operating  limits"  of  the  training  device.  If 
these  limits  are  not  realistic,  the  trainee  should  be  given,  if  pos- 
sible, demonstrations  of  what  happens  at  the  limits  of  control-dis- 
play ranges  on  operational  equipment.  For  example,  what  happens  when 
severe  maneuvers  are  performed  at  near  maximum  aircraft  speeds;  what 
acceleration  occurs  at  sudden  maximum  increases  in  throttle  settings; 
or  the  consequence  of  rapid  reversal  of  thrust  in  landing. 

Such  demonstrations  and  exercises  are  best  given  after  the  trainee 
has  acquired  the  early  procedures  in  operating  an  equipment.  Besides 
providing  perceptual  "anchoi  points"  for  the  trainee,  it  will  take 
care  of  his  tendency  to  want  to  find  out  the  limits  of  what  the  equip- 
ment can  do. 

Trainee  Attitude  And  The  Glamorous  Trainer 


Preliminary  attitudes  of  acceptance  may  be  affected  by  superficial  glitter 
and  realism  of  the  training  device.  Manufacturers,  feeling  under  pressure  for 
general  acceptability  of  their  product,  may  include  expensive  surface  realism 
even  outside  the  work  configuration  of  the  task  to  be  trained. 
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Thun  the  shell  around  seme  train ere  has  been  glamorized  with  gleaming  chrome 
and  wings  and  other  gadgets  irrelevant  to  the  training  of  responses. 

Let  us  remember  that  many  hot-rod  enthusiasts  develop  much  pride  in  a 
piece  of  equipment  encased  in  bent  tin.  Certainly  motivations  do  not  need 
to  be  accepted  as  they  are  found.  Suitable  indoctrination  of  trainees,  in- 
structors, and  high  officials  associated  with  the  purchasing  program  may  re- 
duce the  need  for  surface  glamor. 

After  the  trainee  has  passed  through  his  first  impressions  of  the  equip- 
ment and  got  down  to  actual  work  on  it,  he  will  begin  to  form  attitudes  of  re- 
spect towards  the  amount  of  work  and  effort  which  the  training  device  demands 
of  him.  This  attitude  will  be  os  much  a result  of  method  of  training  as  of 
the  training  device.  The  trainee  may  rapidly  come  to  feel  (ri^itly  or  wrongly) 
that  this  task  is  a pushover,  something  he  can  coast  or  bluff  his  way  through. 
Or  he  may  be  impressed  by  the  need  for  mobilizing  his  maximum  effort  to  meet 
the  training  challenge.  In  any  event,  his  respect  for  the  trainer  will  come 
from  his  confidence  in  it  to  train  him,  the  reliability  of  the  trainer,  his  in- 
teractions with  the  instructor,  and  his  response  to  the  general  businesslike 
atmosphere  of  the  training  program. 

The  outside  of  the  trainer  should  appear  of  workmanlike  construction  and 
be  easy  to  keep  clean. 


INTERMEDIATE  STAGES  OF  TASK  LEARNING 


At  this  point  we  may  think  of  the  trainee  as  entering  a Stage  Two  of  learn- 
ing the  total  job  to  a high  degree  of  operational  skill.  He  has  learned  the 
purposes  of  the  equipment,  the  nomenclature  and  locations  of  displays  and  con- 
trols, some  search  and  scanning  methods  (which  need  to  be  improved  and  stabilized), 
and  has  at  least  moderate  skill  in  moving  controls  in  the  right  direction  when 
given  signals  are  displayed  to  him.  He  has  even  begun  to  make  "patterns  of 
response"  to  complex  stimulus  situations,  but  coordination  and  timing  are  still 
uncertain,  and  reaction  times  are  long. 

Much  of  his  perception  of  signals  and  motor  response  is  interpreted  and 
guided  from  within  himself  by  verbal  or  other  forms  of  "voluntary"  mediation. 

Rather  than  being  controlled  in  large  part  by  automatic  response,  his  perfor- 
mance is  complicated  and  often  confused  by  many  minor  decisions*  Since  he  has 
not  been  under  heavy  time  pressure,  however,  these  behavioral  complications 
have  not  seemed  very  important.  He  is  like  the  novice  automobile  driver  in  & 
dual  control  car  who  must  think  aloud  to  himself  the  individual  foot  and  hand 
movements  which  are  required  to  get  the  car  moving.  He  has  been  in  the  pro- 
tective custody  of  a skilled  driver  who  is  prepared  to  take  over  the  control 
of  the  car  anytime  the  demand  of  the  task  become  overpowering.  However,  he 
hra  not  developed  enough  of  the  skill  rudiments  to  do  ary  driving  in  traffic. 
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The  next  major  activities  will  be  learning  of  precision  and  coordination 
of  movements,  serializing  responses,  perceiving  information  in  larger  blocks 
of  information  in  a unit  of  time,  and  the  beginnings  of  responding  to  ratfcB 
of  target  movement.  Time  pressure  now  does  begin  to  become  imposed  as  part  of 
the  route  to  the  actual  time  demands  of  operations. 

This  stage  of  learning  a complex  task  may  be  marked  by  considerable  sub- 
jective confusion  and  response  interference  effects.  These  difficulties  arise 
from  lack  of  temporal  integration  of  the  processes  of  perception,  mediation 
and  response  execution!  Tnis  confusion  and  internal  habit  interference  will 
drop  out  as  more  and  more  processes  become  semi-automatic  and  automatic}  never- 
theless, these  habit  changes  may  be  very  disruptive  if  pressure  is  exerted  on 
the  trainee's  learning  rate.  One  of  the  Bigna  that  this  stage  1b  concluded  is 
that  the  trainee  is  able  to  perceive  a group  of  signals  as  a pattern  rather 
than  individual  items,  as  when  a telegraphor  receives  by  words  rather  than  by 
letters,  and  by  phrases  rather  than  by  words.  The  reader  who  is  able  to  scan 
ahead  of  the  words  he  is  uttering  has  reached  this  level  of  ability. 

With  respect  to  engineering  simulation,  we  will  find  it  increasingly  im- 
portant to  copy  the  precise  location  of  display  and  control  positions  and  time 
relationships.  Considerable,  thought  must  be  given  now  to  the  "programs,"  or 
organization  of  actual  stimuli  in  time,  whioh  present  the  concrete  problems 
the  trainee  has  to  solve  in  practice  exercises.  If  these  programs  are  too  dif- 
ficult and  too  extended,  the  trainee  will  become  fatigued,  excessively  confused, 
and  discouraged— with  impairment  of  learning  efficiency.  On  the  other  hand,  the 
trainee  should  have  sufficient  challenge  presented  by  these  programs  to  force 
various  of  his  stimulus-response  systems  to  become  more  and  more  automatic  so 
that  his  decision-making  channels  are  not  overlooked. 

It  is  obvious  that  no  general  formulas  or  codes  can  be  prescribed  for  the 
training  equipment  designer,  especially  at  these  stages  of  intermediate  degrees 
of  skill.  As  in  the  early  stages  of  learning  "part-task  trainers"  may  be  valu- 
able, but  their  design  becomes  more  difficult  if  habits  learned  on  them  are  to 
be  successfully  integrated. 

We  might  first  take  an  overview  of  the  variables  and  phases  of  middle 
stages  of  learning  and  then  make  some  comments  pertinent  to  training  devices 

Perceiving  Information  In  Larger  Blocks  In  Space-Time 

The  perceptual  process  seems  to  develop  patterns  of  response  to  stimuli. 

The  kind  of  patterns  and  their  magnitude  probably  depend  in  part  on  the  con- 
ditions of  training  and  in  part  on  the  kind  of  decision  or  critical  motor  re- 
sponse which  habitually  gets  made  to  the  stimulus  clusters. 

There  may  be  at  least  two  processes  at  work  in  the  development  of  per- 
ceptual patterning}  one  of  these  is  in  abstracting  process,  the  other  is  a 
synthesizing  process.  We  can  take  them  up  in  turn: 
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Increaoing  ease  in  dio criminating  task-relevant  from  tank -irrelevant 
cues:  This  takes  place  in  reading  when  we  cease  to  note  the  special  type  faoe  or 
kind  of  typography  used  when  we  read  messages.  Other  examples  include  the 
greater  ease  with  which  radar  operators  can  identify  given  cities  or  terrain 
items  despite  visual  noise;  the  speed  and  certainty  with  which  reconnaissance 
pilots  can  identify  convoys  or  artillery  emplacements,  or  navigators  can 
check  visual  terrain  with  map  landmarks.  A related  example  is  the  pilot  who 
learns  to  check-read  only  a given  pattern  of  instruments  which  are  primary 
during  a given  type  of  maneuver. 


TRAINER  RKXKMENDATIONS  FOR  LEARNING  TASK-RELEVANT  FROM  TASK -IRRELEVANT  CUES 

1.  Artificial  enhancement  of  critical  cues:  In  order  to  speed  up  this 
phase  of  perceptual  skill,  present  to  the  trainee  practice  programs 
in  which  the  abstracting  and  enhancing  of  critical  or  sufficient 
aufts  in  done  artificially. 

2.  Alternate  enhancement  vs  non-enhancement:  Alternate  practice  on 
artificially  abstracted  programs  and  realistic  programs  of  the  same 
content  without  artificial  enhancement. 

3.  Putting  critical  cues  in  complex  backgrounds;  Give  practice  on  a 
wide  variety  of  "noisy1  contexts. 

Example  suggestions:  Training  reconnaissance  pilots  to  detect  truck 
convoys.  Assume  motion  pictures  to  be  the  medium  of  presentation. 

a.  If  possible  determine  what  the  best  of  the  trained  recon- 
naissance men  can  Spot  and  identify  under  the  most  difficult 
visual  conditions.  This  should  suggest  some  of  the  limits 
beyond  which  sample  exercises  may  not  be  very  profitable. 

b.  Assemble  training  exercise  material  ranging  from  easy-to- 
detect  to  difficult-to-detect  target-environment  condit- 
ions. (The  responses  of  sample  trainees  should  test  the 
material  for  difficulty  of  detection  of  critical  cues.) 

c»  In  the  presentation  to  the  -trainee  of  exercises,  follow  a 
period  of  allowing  the  trainee  to  find  the  target  with 
brief,  intermittent  target  enhancement  pictures. 

do  After  artificial  target  enhancement  in  a given  exercise, 
require  the  trainee  to  identify  the  same  target  from  dif- 
ferent approaches,  different  altitudes,  in  different  visual 
noise,  or  to  other  variations  in  factors  important  to  re- 
connaissance viewing. 

The  same  suggestions  would  apply  to  radar  and  naviga- 
tion training. 
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Identifying  a whole  pattern  by  perceiving  any  part  of  it»  Another  basis 
on  which  information  can  with  practice  be  perceived  in  larger  blocks  is  that 
the  trainee  becomes  able  to  identify  a total  stimulus  complex  by  identifying 
only  a portion  of  that  complex.  An  example  would  be  responding  to  the  let- 
ters "trggr"  as  "trigger."  This  form  of  perception  speeds  up  the  possibili- 
ty of  over-all  or  main  response  to  complex  cue  situations. 

Such  patterning  of  response  obviously  eliminates  some  sources  of  mess- 
ages redundancy  in  cues.  But  where  critical  cues  are  embedded  in  noise 
(such  as  is  produced  in  various  operational  programs)  the  trainee  may  be  at 
a disadvantage  if  he  has  not  practiced  learning  to  perceive  critical  cues  in 
noisy  channels  and  thus  be  able  to  use  such  message  redundancy  as  may  be  avail- 
able. 


Training  devices  should  probably  have  the  capacity  to  present  relatively 
pure  signals  which  are  critical  to  job  performance;  but  practice  on  such 
"pure"  signals  should  be  interspersed  vith  practice  on  the  total  stimulus  con- 
text. 


TRAINER  RECOMMENDATIONS  FOR  LEARNING  TO  IDENTIFY  WHOLE  PATTERNS 

Programs  of  operational  "noise"  in  perception  must  be  obtained  through 
a careful  analysis  of  the  job  and  the  environments  in  which  it  must  be  per- 
formed, The  following  suggestions  are  therefore  only  samples  for  illustrative 
purposes. 

1.  Synthetic  contact  flying  and  landing  for  pilots:  Intersperse  in 
practice  exercises  poor  visibility,  haze,  smoke,  twilight  and  other 
factors  making  for  difficult  discernment  of  critical  ground  cues, 

2.  Training  in  the  use  of  radio  aids>  Include  some  practice  programs 
of  static,  interrupted  message  continuity,  sub-standard  speech;  oc- 
casional portions  of  misinformation  embedded  in  correct  information; 
thus  requiring  the  trainee  either  to  check  or  to  make  an  inference 
so  as  to  correct  the  fault  in  the  message. 

3.  Radar  operator  training}  A wide  variety  of  noise  effects  should  be 
included  in  at  least  some  practice  exercises.  Such  visual  noise  may 
be  similar  to  that  originating  from  within  the  equipment  when  mal- 
functioning, or  from  the  environment,  including  enemy  countermeas- 
ures. The  extent  to  which  such  noise  should  precisely  simulate 

the  physical  characteristics  of  operational  noise  is  not  known,  but 
it  seems  somewhat  likely  that  simulation  may  be  only  approximate 
and  still  have  ben*  Tidal  transfer  effects  f'or  pattern  perception. 

It.  Navigation  training:  Similar  principles  may  be  applied  to  practice 
exercises  given  in  navigation  training.  Information  presented  to 
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the  trainee  may  be  inadequate  or  occasionally  even  faulty  in  some 
exercises. 

Training  for  real  life  difficulties 

It  is  frequently  noted  that  synthetic  and  classroom  training 
is  inadequate  for  operations  because  tha  trainee  is  ill-equipped 
to  oope  with  other  than  relatively  "perfect”  conditions.  H1b  skills 
have  not  included  adjustment  to  the  always  less-than-ideal  work  con- 
ditions. The  more  that  the  troubles,  difficulties  and  errors  of  the 
operational  world  can  be  introduced  into  training,  especially  its 
later  stages,  the  leas  must  the  trainee  depend  upon  "operational  ex- 
perience" and  luck  to  see  him  through  to  high  operational  skill. 
Perhaps  the  major  liabilities  in  the  real  world  consist  in  getting 
the  right  information  at  the  right  time  jn  order  to  execute  a mis- 
sion. It  is  therefore  highly  important  that  training  expose  the  train- 
ee to  real  work  problems. 

It  is  also  :Lmportant  that  the  trainee  be  warned  in  general  that 
he  must  expect  to  cope  with  such  problems  in  training,  and  that  the 
purpose  is  to  give  him  practice,  not  to  thwart  him  or  "show  him  up". 
The  difficulties  should  therefore  be  graded  so  that  he  has  a better 
than  50-50  chance  {if  he  has  learned  properly)  to  solve  the  problem 
with  which  he  is  presented.  If  he  fails  to  solve  a problem,  he 
should  be  shown  how  he  could  have  solved  it. 

Perceiving  Signals  in  Advance  Of  Initiating  Motor  Activities 

The  skillful  performer  is  one  who  "eye  is  ahead  of  his  hand."  The  greater 
the  skill,  the  more  flexible  the  time  relationship  between  perceptual  observa- 
tion of  a critical  cue  and  the  motor  response  to  it.  This  is  true  in  a contin- 
uous task  such  as  steering  an  aircraft  and  in  a procedural  task  such  as  typing 
or  plotting  interceptor  courses  in  a ground  control  station. 

Practice  may  also  lead  to  perceptions  anticipating  future  stimuli  in  the 
way  that  we  "expect"  to  hear  thunder  when  we  see  a brilliant  flash  of  lightning 
If  intervening  time  intervals  are  marked  by  intermittent  cues  (ticking  of  a 
watch,  passing  of  mileposts,  rattle  of  a machine  gun)  estimates  of  these  time 
intervals  may  become  very  precise  and,  of  course,  learning  them  is  facilitated. 

It  is  highly  adaptive  to  develop  such  perceptual  flexibility  especially 
where  response  must  be  made  to  complex  environments  which  change  at  compara- 
tively high  speed.  The  groundwork  for  such  skill  will  be  laid  through  adequate 
scanning  habits  acquired  from  practice  at  scanning  through  all  stages  of  train- 
ing. The  following  suggestions  provide  a basis  for  increasing  the  rate  at 
which  perceptual  flexibility  may  be  acquired  beyond  that  of  ordinary  practice: 
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The  folloving  statement  are  provisional  recommendations.  The  methods 
and  devices  should  be  used  only  alter  the  trainee  has  had  preliminary  train- 
ing at  a task  and  is  no  longer  fumbling  with  his  responses. 

1*  Dividing  attention:  During  seme  practice  exercises,  force  a division 
cdT the  trainee  * s attention  between  a primary  task  and  one  or  more 
secondary  taskB.  Secondary  task  stimuli  should  be  intermittent 
rather  than  regular. 

2.  Avoid  looking  at  controls:  Do  not  let  the  trainee  look  at  his  con- 
trols while  he  is  activating  them.  (Under  some  circumstances  and 
tasks  this  may  be  inadvisable.  However,  in  a time-stress  task  in- 
volving a keyboard  or  a complex  set  of  switches,  the  operator  should 
at  least  learn  to  look  at  the  next  control  while  he  iB  operating  the 
previous  control.) 

3.  Intermittent  presentation  of  cuob:  Present  the  primary  cues  for  a 
several  second  duration  followed  by  several  seconds  of  darkness, 
thon  several  seconds  of  the  primary  cues,  and  so  forth. 

U*  Suitable  practice  programs t Provide  practice  programs  bo  that  it  is 
possible  for  the  trainee  to  "look  ahead"  in  a manner  similar  to  what 
he  can  do  in  operations.  (The  possible  difficulty  of  doing  this  in 
seme  part-task  trainers  might  be  one  of  the  training  limitations  of 
such  trainers.) 


Training  notes  Suggestive  evidence  generally  points  to 
the  conclusion  that  it  is  almost  useless  to  try  to  teach  the 
perceptual  skills  in  a perceptual-motor  task  independent  of 
the  motor  performance.  It  is  as  if  muscle  movement  were  an 
important  part-  of  the  total  timing  mechanism  in  a given  task. 
Except  to  teach  identification  and  location  of  objects,  the 
part-task  trainer  which  abstracts  the  perceptual  components 
from  all  motor  performance  in  a perceptual-motor  task  is 
not  likely  to  have  much  training  value. 


Coordinating  Motor  Movements 

There  are  several,  types  of  motor  "coordination": 

a.  Coordinating  a perceived  signal  with  a motor  response. 

b.  Coordinating  a pattern  of  motor  responses  to  be  made  at  the 
same  time- 

c.  Coordinating  motor  responses  which  occur  in  a series. 
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All  three  varieties  have  in  common  the  reduction  of  the  mediating  or  vol- 
untary processes  and  the  corresponding  increase  in  automaticity  of  stimulus- 
response  patterns. 


TRAINER  RECCHMENDATIONS  FOR  LEARNING  ADVANCED  COORDINATION  OF  MOVEMENT 

1.  Control  configuration:  With  respect  to  transfer  of  coordinated  mo- 
tor responses,  the  physical  configuration  of  controls  now  needs  to 
be  simulated  with  exactness,  ospacially  if  relatively  small  time  for 
visual  search  for  the  control  is  permitted  in  the  task. 

2.  Control  forces:  There  is  increasing  evidence  that  absolute  magrd- 
tudes  of  control  loadings  are  not  important  to  copy  either  in  pro- 
cedural or  continuous  tasks  if  visual  or  auditory  feed-back  is  avail- 
able. Although  evidence  is  not  substantial  for  the  following  con- 
clusion, it  would  be  Bafe  to  attempt  to  maintain  the  relationships 

of  control  forces  to  each  other  in  those  controls  which  are  tasK- 
shared  and  time-shared.  But  as  in  other  problems  of  engineering 
simulation,  we  should  recall  that  between-unit  variability  of  opera- 
tional equipment  lowers  the  ceiling  on  amount  of  copying  which  is 
worthwhile. 

3.  Adapting  to  differences  in  control  forces:  In  any  event,  the 
strangeness  in  ‘'feel"  of  different  control  forces  while  shifting 
from  training  to  operations  will  be  partly  subjective  because  of 
different  patterns  of  internal  stress.  However,  even  objective  dif- 
ferences tend  to  lose  their  feeling  of  kinesthetic  strangeness  usu- 
ally in  a matter  of  minutes,  especially  if  the  full  range  of  opera- 
tional response  demands  is  practiced  informally.  For  example,  the 
Student  pilot  when  getting  into  the  air  after  extensive  synthetic 
training,  should  be  allowed  to  go  through  a number  of  maneuvers  in 
rapid  succession,  but  without  being  subject  to  evaluational  criti- 
cism. 

Even  if  control  forces  in  a synthetic  trainer  and  operational 
equipment  are  the  same,  the  former  tends  to  be  judged  as  greater. 
Therefore,  the  control  forces  in  the  synthetic  trainer  should  per- 
haps be  somewhat  lesser  over-all  than  they  are  in  the  operational 
equipment.  The  matter  is,  however,  of  doubtful  important  to  trans- 
fer of  training. 


Serial  Motor  Responses 

People  are  actually  talking  about  “serialized  motor  responses"  wh^n  they 
speak  of  "memorizing"  the  lines  of  a play,  a procedure,  or  a musical  coraposi- 
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^fflsnorizaticr."  or  serializing  of  response  occurs  to  the  extent 
that  each  response  becomes  more  and  more  the  sufficient  cue  for  olicJting  the 
next  response  in  the  series.  Bursts  of  very  rapid  groups  of  res pone may 
occur  without  support  of  kinesthetic  feedback  because  of  that  high  rate. 

‘ evertheless,  even  automatic  response  chains  will  inevitably  require  at  least 
periodic  supporting  cues  from  the  environment  or  from  the  voluntary  "intent" 
of  the  person,  or  both. 

A frequent  disadvantage  of  automatic  response  scrieB  is  that  if  the 
trainee  attempts  to  “think  about"  component  responses  while  he  is  doing  them, 
or  about  to  do  them,  the  response  series  blows  up.  Because  of  the  intensi- 
fied awareness  (self-consciousness)  of  what  one  is  doing  when  performing  "for 
keeps"  as  over  against  rehearsals,  we  must  expect  response  chains  even  well- 
learned  in  synthetic  situations  to  become  disrupted  under  test  conditions. 


TRAINER  RECCMMENBATIONS  FOR  LEARNING  SERIAL  MOTOR  RESPONSES 

1.  Serialization  of  motor  response  in  training  may  be  Bpoeded  up  by  in- 
creasing the  reaction  time  demands  to  the  limits  of  what  the  trainee 
can  do  at  any  given  level  of  skill.  ThiB  can  be  done  by  presenting 
programs  which  are  complex  relative  to  the  trainee's  Btage  of  train- 
ing. Ideally,  the  trainer  should  be  flexible  in  permitting  the  in- 
structor to  speed  up  or  slow  down  the  task  demands. 

2.  Transition  from  trainer  to  operational  equipment!  If  a s.  ^l  has 
been  learned  to  considerable  autoniaticity  in  a trainer,  the  shift  to 

an  operational  device  may  be  accompanied  by  temporary  "Belf-consciousness" 
about  making  responses  and  verbalizations  which  will  temporarily  dis- 
turb or  even  disrupt  the  trainee's  ability  to  do  the  task.  Either  this 
kind  of  temporary  disturbance  should  be  recognized,  cr  some  practice 
on  the  trainer  should  be  interspersed  with  practice  on  the  operational 
device.  Without  such  provisions  it  is  possible  that  the  trainer's 
validity  may  seem  to  be  less  than  what  it  might  be. 

3.  Response  chains  may  be  more  quickly  learned  if  the  trainee  ls  encouraged 
to  pre-position  his  hands  and  feet  on  controls  as  quickly  possible, 
even  when  practicing  slow  programs  of  control  activat.’  < n.  For  this 
reason  as  well  as  others,  the  instructor  should  be  ab.~  .v.  view  the 
trainee's  manipulation  of  controls. 


1.  Recall  that  we  are  including  verbal  behavior  as  motor  behavior 
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Warning:  Responses  which  have  become  part  of  a serialized 
chain  are  not  readily  modified  without  disrupting  the  entire 
performance.  If  flexibility  of  response  items  within  a chain 
is  desired,  practice  should  not  be  continued  too  long  on  a rote 
series  which  does  not  represent  operational  variety.  Ono  dan- 
ger of  over-practice  on  a part-task  trainer  is  that  responses 
may  become  serialized  which  in  the  over-all  job  should  be  flex- 
ible and  subject  to  decision  and  "judgment11  by  the  operator. 

This  difficulty  can  perhaps  best  be  overcome  by  interspersing 
part- task  and  total  task  practice. 

Furthermore,  automatized  serial  responses  tend  to  be  an- 
chored to  the  ratos  in  which  the  response  items  were  practiced. 
Change  in  rates  of  component  response  demands  will  tend  to  dis- 
rupt the  entire  chain. 

Responding  To  Rates  Of  Stimulus  Movement 

When  a batter  swings  at  a flung  ball  his  response  is  compounded  of  the 
position  and  inertia  of  the  bat  (which  presumably  he  "knows")  and  of  the  dir- 
ection, velocity  and  position  of  the  ball  (which  hs  estimates).  This  example 
is  the  analogue  of  the  fighter  pilot  flying  an  interception  course  to  an  ev- 
ading enemy  bomber.  Let  us  think  of  the  bat  as  the  "cursor,"  and  the  ball  as 
the  "target."  The  weight,  size,  shape  and  balance  of  the  bat  we  may  call  "cur- 
sor characteristics"  and  the  trajectories  of  the  ball  will  contain  what  we  may 
call  the  "target  characteristics." 


Sample  of  "predictable"  target 
course  (repeated  pattern  like  a 
projectile  path) 


Sample  of  "unpredictable"  target 
course  (fighter  aircraft  dogfight) 


With  — information  the  skilled  operator  can  predict  (with  given  error) 
this  — > much  of  the  target's  future  position. 

mDC  TR  5U-563  4*6- 


Stage  of  learning  and  simulation 
Intermediate  learning 

In  order  to  intercept  or  trade  a targot  with  minimum  error  it  ie  noce38~ 
ary  for  the  operator  to  anticipate  its  future  position  from  an  earlier  segment 
of  data  up  to  the  present  moment.  The  movement  of  some  typos  of  target  is 
highly  predictable  (at  least  theoretically)  from  even  email  segments  of  data. 
The  motions  of  a flung  ball  or  of  other  projectiles  are  examples.  The  move- 
ment of  other  targets  is  less  predictable  (even  theoretically)— for  example, 
self-propelled,  maneuverable  targets  such  as  aircraft  and  guided  missiles  may 
generate  continuously'  changing  patterns. 

Generalizations  about  learning  and  performance  obtained  from  predictable 
target  paths  (tracking  sine  waves  or  continuous  circular  motion)  should  be 
made  only  with  considerable  skepticism  to  non-predictable  target  courses. 


TRAINER  RECOMMENDATIONS  FOR  TEACHING  "PREDICTABLE"  TARGET  MOTIONS 

Seme  recommendations  can  be  made  in  the  preparation  of  "programs"  for 
practice  in  training  to  track  targets  whose  future  positions  are  relatively 
predictable,  and  can  thus  be  extrapolated  from  a small  segment  of  target  data. 

1.  The  control-display  time  characteristics  of  the  "cursor"  (aircraft, 
gunnery  system,  ana  so  forth)  should  be  copies  to  tolerances  of  be- 
tween-unit  variation.  (See  page  26  for  definition  of  "between- 
unit"  variation.) 

The  control-display  ratios  of  the  operational  task  should  be 
highly  simulated  if  the  operational  task  (l)  has  a heavy  time  load  of 
rapid  sequential  responses,  and/or  (2)  a noticeable  control-display 
time  lag. 

2.  Programs  of  target  courses  should  copy  the  rate  and  rate-of-change 
characteristics  of  the  operational  targets.  But  the  programs  should 
8 ample  also  from  target  courses  having  motions  beyond  the  limits  ex- 
pected in  operations.  By  motions  is  meant  both  rates  of  motion  and 
maneuverability.  Environmental  programs  should  sample  from  extremes 
in  severity  in  order  to  provide  the  trainee  with  practice  in  adjust- 
ment to  severe  "noise." 

3*  Cues  available  in  operations  for  predicting  more  precisely  the  course 
of  a target  should  also  be  available  in  synthetic  practice  either 
directly  to  perception  or  at  least  symbolically. 

li.  Supporting  or  context  cues  available  in  operations  for  judging  dis- 
tance, rate  and  target  track  should  be  introduced,  if  possible,  in 
synthetic  training.  If  this  is  not  practicable.-  then  synthetic 
practice  should  be  interspersed  with  operational  practice  (with 
"safe"  targets,  of  course). 
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5.  Unique  solution  problems : Exercises  should  be  programmed  to  include 
interceptor  problems  whose  solution  is,  within  fairly  Bmall  limits, 
unique  with  respoct  to  a correct  course  which  the  trainee  can  select. 
That  is,  the  maximum  maneuvering  and  rate  capacities  of  the  trainee’s 
equipment  can  effect  a kill  in  such  air  exercise  only  if  proper  dir- 
ections and  rates  are  selected  at  every  phase  of  the  attack  pattern. 
Such  practice  will  reduce  the  dependence  placed  by  the  trainee  on 
the  capacity  of  his  equipment  to  out-maneuver  the  target  at  close 
range.  Such  a problem  might  include  fuel  capacity  as  a variable. 

In  order  to  train  an  operator  to  track  highly  erratic  targe Is,  however, 
it  becomes  necessary  to  teach  him  to  extrapolate  future  target  positions  from 
relatively  small  segments  of  track  information.  We  are  presuming  that  smooth- 
ness of  tracking,  at  least  for  a number  of  seconds,  is  as  important  aB  fre- 
quency of  coincidence  of  target  and  cursor  track.  This  presumption  is  based 
upon  the  neod  of  a gun  which  uses  a computer  to  predict  lead  angle.  A minimum 
of  oscillation  or  over-control  is  also  required  of  the  trainee. 

Teaching  the  trainee  to  track  erratic  courses  means  that  "remembered  infor- 
mation" about  target  courses  is  of  relatively  little  use;  it  also  means  that, 
for  a comparable  degree  of  accuracy,  he  must  respond  to  shorter  segments  of 
presented  target  data.  In  short,  his  perceptual -mo tor  response  patterns  will 
have  to  be  stepped  up  in  speed. 

In  general,  a compensatory  tracking  device  such  as  air-to-air  radar,  pre- 
sents to  the  operator  more  erratic  information  than  visual  air-to-air  combat. 

(it  can  be  logically  demonstrated  that  the  addition  of  any  independent 
variable  to  the  motion  of  a target  increases  the  error  of  prediction  based  on 
any  temporal  segment  of  data  generated  by  the  target.  The  greater  the  time 
between  an  event  A and  a prediction  of  a future  event  B based  on  A,  the  greater 
the  error  of  prediction.  Hence,  to  maintain  a given  degree  of  prediction  error 
when  another  variable  is  added  to  target  motion,  the  prediction  must  be  re- 
stricted to  a shorter  time  interval  between  A and  B.  This  means,  for  the  opera- 
tor, a more  rapid  reaction  time  and  less  oscillation  of  adjustments  in  order  to 
maintain  a given  degree  of  tracking  accuracy  or  target  kills.) 

TRAINER  RECCi'MMIlATIONS  FOR  TEACHING  "ERRATIC"  TARGET  MOTIONS 

1,  Range  of  target  v ograms:  Skill  in  tracking  (relatively)  erratic 
targets  will  be  acquired  from  highly  motivated  practice  at  a large 
variety  of  erratic  target  course  exercises.  Target  patterns  in  these 
expi  ises  should  probably  exceed  the  capacity  of  known  targets  to 
moYw  in  erratic  patterns;  thus  they  need  not  simulate  the  motion 
characteristics  of  operational  targets. 
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Because  the  skill  obtained  from  this  kind  of  practice  vill  be 
of  a higher  or  more  di pficult  order  than  skill  in  tracking  identifi- 
able and  predictable  target  motions,  provision  should  be  made  for 
far  more  practice. 

2.  Interspersing  simulated  operational  target  programs*  Sane  practice 
should  be  provided  occasionally  on  target  courses  which  at  least  ap- 
proximate those  which  could  be  generated  by  actual  enemy  targets. 

3.  Variety  of  target  programs?  Exercises  within  a practice  session 
should  contain  a wide  variety  of  erratic  courses  so  as  to  avoid  the 
development  of  any  stable  "set"  to  respond  in  a given  way.  Such  vari- 
ation will  also  help  determine  if  the  trainee  is  developing  consist- 
ent error  habits.  The  target  courses  should  include  some  relatively 
easy  patterns,  not  only  for  motivational  purposes,  but  to  allow  the 
trainee  occasionally  to  "see  what  it  looks  like  to  be  on  a target.” 

An  "easy"  target  course  is  one  approximating  a straight  line,  and  in 
which  the  target  moves  slowly  relative  to  the  adjustment  speed  pos- 
sible to  the  cursor  and  the  operator. 

1|.  Order  of  target  course  presentation:  The  order  of  target  courses 
should  not  be  repeated  within  or  between  practice  sessions.  Ideal- 
ly the  trainee  would  never  run  the  same  target  course  more  than  once, 
except  for  special  demonstration  purposes.  (See  note  below  for  other 
exceptions.)  The  reason  is  to  rule  out  his  memorizing  the  courses. 
Practically,  it  is  usually  necessary  to  repeat  target  courses  in  suc- 
ceeding exercises  unless  the  target  course  generator  is  not  actually 
a second  instructor  himself.  It  is  possible  that  a complex  tracking 
skill  could  be  learned  from  50  to  100  non-repeat ed  courses  without 
memorization  of  courses  occurring. 


(NOTE:  There  is  some  logical  possibility  that  presenting 
each  target  course  twice  in  immediate  succession  would  bene- 
fit the  trainee  at  least  early  in  training.  Presumably  he 
would  be  able  to  incorporate  target  knowledge  with  cursor 
knowledge  and  have  the  advantage  (on  the  second  run)  of 
seeing  a "success".  The  practical  disadvantage  would  be 
that  the  trainee  might  feel  he  always  had  a second  chance, 
and  would  not  try  very  hard  on  the  first  chance  at  a given 
target  course.) 

5*  Using  response  context;  In  teaching  a trainee  to  track  targets,  it  is 
essential  to  embed  the  exercise  between  (a)  the  activity  just  preced- 
ing the  tracking  and  (b)  the  activity  immediately  following  the  track- 
ing of  a critical  target.  Putting  the  tracking  in  the  job  context  is 
especially  desirable  if  precise  timing  is  demanded  of  the  activities, 
and  of  the  shift  from  one  activity  to  another.  (Recall  that  a re- 
sponse is  determined  in  part  by  stimuli  acting  at  the  moment  and  in 
part  by  stimuli  expected  in  the  future).  The  fighter  pilot  who  has 
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no  more  than  two  or  throe  seconds  in  which  to  pull  out  of  a collis- 
ion course  after  firing  will  have  to  learn  to  avoid  anticipatory  re- 
sponses which  will  disturb  his  perceptions,  timing  and  aim.  A train- 
ing device,  free  from  operational  hazard,  is  appropriate  for  learn- 
ing to  integrate  temporal  patterns  of  response  without  these  patterns 
interfering  with  each  other.  Wo  can  realistically  expect  that  the 
transfer  from  a non-hazardous , synthetic  situation  to  the  hazards  of 
operations  will  in  itself  have  some  disturbing  effect  on  the  aiming 
responses.  Such  effects  will  bo  less  disturbing  and  more  rapidly 
adapted  to  if  aim  followed  by  properly  timed  pull-out  has  been  prac- 
ticed in  the  safe  situation. 


In  summary,  practice  in  a time-critical  task  should  include 
activities  which  immediately  precede  and  immediately  follow  (espec- 
ially the  latter)  the  critical  performance  of  the  task. 


HIGH  STAGES  OF  TASK  LEARNING 


We  now  will  deal  with  what  are  obviously  the  stages  of  higher  learning 
of  tasks  and  skills.  The  shift  from  middle  degrees  of  learning  to  the  high 
degrees  of  Bkill  sometimes  is  shadowy  and  practically  impossible  to  discern 
in  any  actual  trainee  except  over  several  weeks.  On  the  other. hand,  there 
may  be  sudden  spurts  of  maintained  improvement  quite  evident  in  a short  time. 

A graduate  pilot  expressed  in  simple  language  the  arrival  of  this  degree  of 
competence  by  saying:  “When  you  are  learning  how  to  fly,  you  have  to  think 
not  only  about  what  you  are  supposed  to  do,  but  also  how  you  are  going  to  do 
it.  When  you  really  know  how  to  fly,  all  you  have  to  do  is  think  what  you  want 
to  do  and  the  airplane  does  it.” 

An  important  distinction  of  terms  must  be  made.  By  "high  degree  of  learn- 
ing" a skill, is  meant  high  degree  for  some  given  individual  who  has  been  prac- 
ticing a complex  job  or  task.  We  do  not  here  mean  a high  degree  or  level  of 
skill  compared  to  some  other  persons.  Thus  even  the  poorest  pilot  of  the  lot 
will  have  acquired  "automatic  habits,"  "anticipations,"  "coordinated  response/1 
and  so  forth.  The  quality  of  his  performance  compared  to  a criterion  may  be 
poor,  but  the  psychological  processes  nevertheless  will  tend  to  take  place. 

VJe  may  also  find  large  individual  difference  between  rate  of  learning  at 
different  stages  of  acquiring  a set  of  skills.  One  trainee  may  be  a poor  rote 
learner,  so  he  may  be  among  the  slower  trainees  in  the  early  portions  of  a 
training  course..  On  the  other  hand,  he  may  bring  to  training  excellent  per- 
ceptual-motor coordinations,  so  that,  if  he  does  not  get  washed  out  in  early 
training,  he  may  be  among  the  better  trainees  in  the  later  stages  of  train- 
ing. 

The  principal  developments  in  very  high  stages  of  skill  are  that  (a) 
responses  become  rapid,  smooth  and  automatic;  (b;  the  operator  is  better  able 
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to  anticipate  and  prepare  for  •’future'1  events  and  becomes  proficient  in  por- 
iorming  to  wider  ranges  of  complex  programs*  These  developments  are  partly 
due  to  the  increased  "channel  capacity11  of  the  operator  arising  fror  less 
need  for  him  to  think  about  specific  movements  he  must  next  perform. 

Automatized  Habits 


Habits  are  automatized  to  the  extent  that  responses  occur  to  stimuli 
without  the  intervention  of  self  instruction— that  is,  without  having  to 
think  about  them,  such  as  blinking  when  an  object  approaches  an  eye,  or  the 
experienced  driver  throwing  on  the  brakes  when  an  obstacle  looms  close  ahead. 

Behavior  is  usually  automatic  only  to  a degree.  Groups  of  responses  way 
be  emitted  automatically,  but  voluntary  control  (at  least  in  part)  may  be 
needed  to  direct  what  next  group  of  responses  will  occur.  For  example,  when 
a driver  first  learns  to  shift  gears,  he  will  think,  in  turn,  of  the  individual 
response  of  depressing  the  clutch,  releasing  the  accelerator,  shifting  the  gears, 
and  so  forth.  With  sufficient  of  the  right  kind  of  practice  these  individual 
responses  become  organized  into  what  we  might  call  the  "act"  of  gear-shifting. 

By  proper  training,  this  act  will  be  capable  of  variation,  such  as  shifting 
gears  on  an  uphill  grade.  Because  a response  is  part  of  a larger  behavioral 
unit  does  not  rule  out  its  being  under  voluntary  control  in  another  context. 

The  powerful  tendency  to  slam  on  the  brakes  when  a car  is  heading  into  danger 
can  be  inhibited  by  voluntary  effort,  as  it  must  be  on  icy  roads.  The  "mental" 
effort  and  vigilance  may  sometimes  have  to  be  very  considerable,  however,  to 
be  successful  against  a strongly  learned  response. 

It  can  readily  be  seen  that  when  responses  are  becoming  automatic,  they  are 
less  flexible  with  respect  to  changes  in  stimulus  conditions.  The  operator  may 
seem  to  behave  stupidly,  although  rapidly,  to  the  specific  cue  which  triggers 
the  automatic  response.  We  can  therefore  advise  that  when  responses  are  becoming 
automatic,  practice  should  be  given  In  the  full  range  of  urograms,  or  stimulus 
environments,  which  have  a bearing  on  those  response  systems.  Before  slamming 
on  the  brake  becomes  completely  automatic  to  any  "emergency"  situation,  the  driver 
should  be  given  practice  on  the  proper  braking  for  slippery  roads,  or  for  going 
around  curves  at  critical  speeds,  or  for  going  downhill.  Thus  he  learns  to  dis- 
criminate when  one  kind  of  brake  response  is  required  rather  than  another,  instead 
of  the  wasteful  process  of  unlearning  with  its  often  unreliable  results  under 
stress. 

This  consideration  is  obviously  important  to  the  planners  of  training  pro- 
grams especially  if  part-task  trainers  and  other  synthetic  trainers  are  used. 
Although  many  responses  must  become  integrated  into  automatic  habit  patterns 
to  make  skill  in  complex  performance  possible,  it  is  at  this  stage  that  the 
greatest  risk  of  both  inefficiency  and  invalidity  of  training  may  arise.  Being 
independent  of  generalization  through  mediating  processes,  automatic  responses 
tend  to  be  triggered  by  only  those  stimuli  presented  in  practice,  and  all  those 
stimuli  presented  in  practice.  The  trainee  may  have  learned  to  slam  on  the 
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brakes  at  ar iy  sudden  and  unusual  stijnulus  or  situation  occurring  widle  driv- 
ing. In  one  sense,  this  response  may  bo  too  specific  in  that  if  a stimulus 
condition  develops  gradually,  ho  will  not  brake  even  when  he  should.  On  the 
other  hand,  the  response  may  be  too  general  in  that  some  sudden  changes  in 
stimulus  conditions,  such  as  the  squeal  of  tires  and  tho  lurch  of  a car  around 
a sharp  turn  at  high  speed,  do  not  call  for  slamning  on  the  brakeB. 

Thus  we  can  see  how  improper  part-training  can  result  in  inappropriate 
kinds  of  response  automatization. 

We  can  increase  the  rate  at  which  responses  become  automatic  by  imposing 
the  following  practice  conditions: 

1.  Highly  repetitive  practice  with  short  rest  intervals. 

2.  Practice  under  heavy  load,  that  is,  making  the  trainee  pay  attention 
to  many  things  (or  even  extraneous  things)  while  performing  the  cri- 
tical response  patterns.  This  will  be  effective  only  if  the  trainee 
has  learned  the  task  to  a moderate  degree. 

3.  Practice  when  the  trainee  is  under  heavy  anticipations ; these  may  in- 
clude anxiety  and  stress  patterns.  This  condition  Is  related  to  con- 
dition 2 above. 

li.  Practice  at  speeds  which  tax  the  trainee  !s  momentary  capacities  to 
perform  the  responses. 

The  common  factor  underlying  all  these  conditions  is  that  the  trainee  is 
distracted  from  thinking  about  the  responses  he  is  having  to  make  while  still 
being  forced  to  make  uhem. 

The  preceding  principles  imply  that  training  devices  intended  to  train 
for  response  automatization  must  be  capable  of  presenting  programs  of  consid- 
erable variety,  complexity  and  .rates  of  presentation. 


TRAINER  RECOMMENDATIONS  ON  AUTOMATIZING  HABITS 

1.  Providing  against  memorizing  practice  stimuli t Wide  variation  in 
target  course  patterns  are  imperative,  because  the  advanced  trainee 
will  have  better  capacity  to  notice  and  remember  target  and  signal 
data.  (No  longer  having  to  "think"  about  his  motor  response,  he  can 
direct  more  of  his  attention  to  the  environmental  stimulus  features.) 

2.  Emphasis  on  habits  of  double  checkings  Training  should  utilize  the 
capacity  for  greater  flexibility  of  the  advanced  trainee's  attention 
by  drilling  habits  of  double  checking  of  perception  (signal  inter- 
pretation) and  of  action  (proper  control  activation). 
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The  instructor  in  probably  the  best  instrument  for  training  to 
double-check  signal  interpretation  and  response  pre-positioning 
and  output.  This  end  can  bo  achieved  through  directions  plus 
trainee  motivation  to  perform  the  task  at  higher  degrees  of  ac- 
curacy and  reliability.  The  idea  of  reliability  of  performance 
is  difficult  to  get  across  to  trainees  until  after  they  have  had 
sane  accidents  or  close  shaves  due  to  their  response  unreliability. 
It  is  better  that  the  trainee  experience  such  unreliability  in 
the  training  equipment.  Provision  might  be  made  for  signaling 
cue  to  suddenly  change  after  the  trainee  has  checked  it  once  and 
shifted  his  attention  to  some  other  cue.  Thus  if  the  trainee  did 
not  double  check  the  first  cue,  he  would  not  respond  correctly. 

This  type  of  program  is  an  attempt  to  simulate  the  operator's 
making  an  incorrect  perception  and  then  acting  on  the  basis  of 
incorrect  data  becmise  he  did  not  double-check  it. 

There  is  an  advantage  to  the  operator's  monitoring  his  own 
inputs  and  outputs.  By  so  doing  he  acquires  and  maintains  a 
higher  level  of  voluntary  control  carer  his  habits.  He  prevents 
them  from  becoming  so  automatic  that  he  is  aware  even  of  their 
occurrence  or  non-occurrence.  In  other  words,  self -monitoring 
reduces  the  tendency  for  repetitive  activities  to  become  entire- 
ly mechanical  with  the  almost  inevitable  degradation  which 
mechanical  habit  systems  undergo.  In  popular  language,  self- 
monitoring enables  the  operator  to  keep  "knowing  what  he  is 
doing." 

Self -monitoring  will  not  tend  to  arise  as  a spontaneous  set 
of  habits.  Even  if  self -monitoring  does  develop  spontaneously, 
it  may  not  be  efficient.  For  these  reasons  explicit  provisions 
should  be  made  for  training  of  self -monitoring  habits. 


Anticipatory  Response  And  Time  Intervals 

Anticipatory  modes  of  response  described  in  earlier  stages  of  learning 
will  become  increasingly  more  reliable.  The  trainee  will  probably  have 
learned,  among  other  things,  the  optimal  segment  of  target  information  to  as- 
similate per  unit  of  motor  response,  and  this  segment  will  be  perceived  so  as 
to  allow  suitable  reaction  time  to  the  trainee.  This  situation  is  somewhat 
similar  to  that  of  the  person  skilled  in  reading  aloud  who  learns  to  "take  in" 
the  right  number  of  words  at  a time,  and  to  apprehend  them  with  the  right  amount 
of  time  ahead  of  his  tongue  movements. 


TRAINEE  RECOMMENDATIONS  ON  DISCONTINUOUS  VS.  CONTINUOUS  TASKS 

1.  Simulation  in  discontinuous  tasks?  "When  the  trainee  is  reaching 
hagh  degrees  of  response  automaticity  in  discontinuous  tasks,  it  is 
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of  greatest  important  to  simulate  locations  of  controls  and  dis- 
plays, display  patterns  as  they  occur  in  space  and  in  time,  and  any 
time  delays  between  control  activation  and  feedback  cues  which  call 
for  a next  response  in  a series. 

If  the  response  sequences  are  task-paced,  the  paoing  of  the 
trainer  should  copy  those  of  the  operational  device  or  preferably  be 
somewhat  faster,  especially  if  the  trainee  will  have  heavy  load  or 
stress  in  operations. 

2.  Simulation  in  continuous  tasks;  When  a tracking  task  involving  com- 
plex factors,  including  discontinuous  (procedural)  components  such 
as  switch  pressing  or  voice  communications,  is  becoming  highly  auto- 
matized it  is  essential  to  copy  programs  of  displays.  Such  programs 
should  take  into  account  what  the  operator  has  to  discriminate  rath- 
er than  all  the  environmental  factors  which  may  be  present. 

Example:  If  the  change  in  horizon  curvature  with  difference  in 
elevation  is  not  used  as  a critical  cue  in  any  task  the  pilot 
performs,  then  it  would  be  unnecessary  to  duplicate  this  envir- 
onmental factor  in  a visual  contact  pilot  trainer. 

3,  Risplay-Control  Sensitivity  of  the  trainer  should  not  be  greater  than 
that  of  the  operational  e'quipment—a  frequent  characteristic  of  train- 
ers. Over  sensitivity  of  tho  trainer  is  unduly  demanding  of  the 
trainee^  attention  during  practice;  it  may  also  give  the  trainee  un- 
realistic expectations  of  the  response  capability  of  the  operational 
equipment  in  maneuverability.  (Sensitivity  is  the  smallest  movement 
of  a control  which  results  in  a display  change. ) 

Control  pressures  and  control-display  ratios  will  be  somewhat 
less  critical  than  the  copying  of  the  characteristic  lags  between 
control  activation  and  display  response,  even  in  the  higher  stages  of 
training. 


TRAINER  RECOMMENDATIONS  FOR  LEARNING  ANTICIPATOR!  RESPONSE 

1.  Copy  time  intervals:  If  time  intervals  must  be  estimated  by  the 
operator  as  part  of  his  task  (as  in  visual  estimates  of  interception) 
it  goes  without  saying  that  these  should  now  be  copies  with  high  pre- 
cision, especially  if  the  task  does  not  provide  or  permit  continuous 
supporting  cues  about  the  passage  of  time.  Often  the  use  of  a chron- 
ometer by  the  operator  may  not  fit  into  the  t:!me— or  other— demands 
of  the  task. 

2.  Ranges  of  target  movement:  The  exception  to  the  statement  above  oc- 
curs in  later  trials  on  a complex  training  device  (before  all  train- 
ing is  taken  over  on  the  operational  device).  Here  it  is  recommended 


-56- 


WADC  TR  5U-563 


Stage  of  learning  and  simulation 

High  learning 


that  the  trainer  should  incorporate  wider  rangeB  of  target  movement 
than  occur  in  operations  with  respect  to  velocities,  accelerations 
and  spatial  patterns. 

3*  Increased  feedback  information  to  the  trainee:  In  these  late  stages 
of  learning,  the  trainee  is  in  the  best  condition  for  responding  to 
precise  knowledge  of  results  information.  Because  of  increased  auto- 
maticity  of  response  and  increased  perceptual  flexibility  (the  lat- 
ter due  to  his  perceiving  ahead  of  his  motor  response),  he  has  "men- 
tal channel  capacity"  to  absorb  additional  information;  and  this  in- 
formation may  tales  the  form  of  knowledge  of  results.  Learning  feed- 
back may  be  presented  to  the  trainee  verbally  by  the  instructor,  or 
through  special  equipment  channels  devised  for  the  purpose.  The 
trainee  can  now  assimilate  such  information  with  less  risk  of  blow- 
ing-up on  what  he  is  doing  at  the  moment  than  he  could  during  the 
intermediate  stages  of  skill  formation. 

Thus  the  advanced  trainee  could  be  given  out-of-tolerance  data 
at  about  the  same  time  with  respect  to  trim,  throttle  position,  al- 
titude and  direction.  The  less  advanced  trainee  might  be  able  to 
cope  with  only  two  such  factors  at  a time. 

Furthermore,  whereas  the  less  advanced  trainee  might  be  given 
heading  information  in  $ degree  units,  the  more  advanced  trainee 
could  cope  with  heading  information  in  1 degree  units. 


Responding  to  Operational  Stress 

Before  a trainee  is  operationally  proficient  he  must  be  a reliable  per- 
former in  situations  where  he  "plays  for  keeps".  It  is  the  difference  between 
the  rehearsal  and  the  live  show;  landing  a simulator  and  landing  the  aircraft 
on  the  deck;  shooting  at  a synthetic  pip  and  shooting  at  a pip  whose  origin 
can  shoot  back. 

We  may  think  of  stress  as  internal  stimulation  which  competes  with  the 
critical  stimulus-response  processes  required  by  operations.  It  is  a form  of 
"noise"  in  the  operator's  private  system.  It  tends  to  be  somewhat  more  de- 
structive of  verbal-conceptual  processes  than  of  perceptual-motor  processes, 
especially  if'  the  latter  are  response-cued,  that  is,  triggered  off  by  previous 
responses  in  a series. 

The  training  implication  is  that  we  may  get  the  trainee  used  to  heavy 
loads,  such  as  are  implied  by  internal  stress,  by  imposing  heavier  task  loads 
on  the  trainee  during  training  than  he  may  usually  get  in  operations.  That 
is,  in  later  training  we  should  provide  environmental  programs  of  such  com- 
plexity that  the  trainee  becomes  used  to  working  under  heavy  signal  loads. 
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Situational  stress  usually  becomos  reduced  after  the  first  few  moments 
during  which  the  operator  is  active  in  operations.  It  is  like  the  rapid  wear- 
ing off  of  self-consciousness  of  a speaker.  This  happens  unless  the  perfor- 
mance breaks  down  altogether.  For  this  reason  it  is  good  to  concentrate  somo 
high  degree  of  practice  on  the  first  portions  of  ejctonded  job  cycles.  This 
should  be  done  on  the  trainer  especially  preceding  the  trainee^  transition  to 
operational  equipment  when  we  can  expect  his  nervousness  to  be  greatest  and 
confidence  at  its  lowest. 


TRANSITION  TRAINING 

Transition  training  is  the  link  between  an  already  learned  skill  on  one 
kind  of  equipment  and  skill  in  operating  a new  but  related  kind  of  equipment. 
Major  differences  between  tasks  may  arise  through  (a)  general  acceleration  of 
rates  at  which  perceptual-motor  and  decision-making  response  are  called  for 
(as  in  shifting  from  propeller-driven  to  jet  aircraft  especially  in  landing 
operations)  (b)  when  reversals  of  discrete  perceptual-motor  habits  are  re- 
quired, and  (c)  when  new  task  aspects  are  added  (tail-pipe  temperature  in 
jets),  or  tasks  acquire  new  or  increased  importance  (fuel  management  in  jets). 

Carefully  prepared  task  analyses  of  the  old  and  new  jobs  should  be  made  on 
parallel  formats.  The  following  kinds  of  differences  should  be  noted  as  tran- 
sition problems: 

1.  Special  note  should  be  taken  of  the  different  kinds  of  anticipa- 
tions demanded  by  the  new  task  to  patterns  of  cues  which  also 
appear  in  the  old  task. 

2.  Learning  new  procedures,  or  an  old  procedure  which  has  to  be  per- 
formed in  a different  way. 

3.  Interpreting  old  displays  in  a different  way,  as  for  example 
when  different  stall  speeds  occur  in  the  old  and  new  tasks. 

h.  Modification  of  all-or-none  responses,  or  responses  which  have 
long -delayed  rather  than  immediate  feedback. 

5*  Noticing  special  signals  in  the  new  task,  especially  if  these  or 
similar  signals  appeared,  in  the  old  task  but  did  not  call  for 
any  response. 

6.  Special  discontinuities  of  critical  values  in  the  per for mance  of 
new  operational  equipment. 

Differences  in  ^critical  discontinuities"  (item  6)  will  be  especially  hard 
to  learn  in  transition  training  for  the  same  reason  that  it  is  hard  to  change 
one  response  in  an  otherwise  similar  pattern  of  responses.  That  is,  when  we 
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have  learned  a response  series  like  a,b,c,d,e,f,  there  is  difficulty  in  learn- 
to  say  (quickly)  a,b,c,p,f,  or  make  other  substitutions  of  one  or  two  items. 

It  would  be  especially  hard  to  learn  "a,b,d,c,o,f,"  The  longer  the  series 
which  has  been  learned  as  a pattern  or  unit,  the  more  difficult  to  learn  to 
make  a substitution,  omission,  or  addition  of  response  items. 

Training  should,  of  course,  concentrate  practice  on  these  differences. 

But  after  rather  little  "procedural11  practico,  the  practice  should  be  contin- 
ued (for  the  sake  of  efficiency)  in  the  full  context  of  task  performance  in 
at  least  the  physical  configuration  of  the  equipment  around  the  operator.  By 
"full  context"  is  meant  the  various  behaviors  which  go  on  at  about  the  same 
time  in  whatever  time  segment  of  the  job  is  being  trained. 

To  put  it  another  way,  it  does  not  seem  as  if  part-task  trainers  are 
likely  tc  have  much  pay  off  for  transition  training  beyond  very  elementary 
stages  of  the  learning  of  related  tasks  requiring  different  modes  of  re- 
sponse. ^ On  the  other  hand,  part-task  trainers  might  be  as  useful  as  in  othGT 
training  situations  for  the  teaching  if  new  tasks  for  which  there  was  no  cor- 
relate in  the  old  job. 

A special  note  should  be  added  about  transition  training.  The  trainee 
may  show  considerable  proficiency  in  the  new  task,  but  the  passage  of  time 
without  practice  on  the  new  task,  or  the  experience  of  heavy  load  or  stress 
in  operations,  may  cause  "regression"  to  old  habit  systems.  For  these  reasons 
it  is  wise  to  provide  for  some  practice  beyond  evidence  of  operational  profi- 
ciency. 

Practice  on  habit  correction  should  be  done  on  intermittent  trials  rather 
than  confined  to  a single  block  of  trials  at  one  time  and  thenceforth  ignored. 

Because  of  the  temporary  unreliability  of  superficially  well-learned 
habits  in  transition  training,  there  should  be  emphasis  on  the  trainee's 
double  checking  his  perceptual  responses  and  his  motor  responses.  The  vigil- 
ance and  insistance  of  the  instructor  is  perhaps  the  best  device  for  such  em- 
piu.sisj  provision  should  therefore  be  made  for  the  instructor  to  view  the 
trainee  for  the  purpose  of  instilling  habits  of  double  checking,  as  well  as 
for  other  reasons. 

NOTE:  The  reader  will  have  noticed  that  the  beginning  parts  of 
this  chapter  were  strongly  oriented  towards  training  equipment, 
but  that  the  middle  and  end  parts  were  more  oriented  towards  pro- 
gramming of  practice  content  and  use  of  training  equipment.  This 
orientation  reflects  the  proper  concern  in  the  early  stages  of 


1 Unless  some  time  segment  of  job  activities  is  defined  as  a "part-task." 
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learning  with  the  equipment  supports  for  training  and  the  con- 
cern in  the  later  stages  of  learning  with  the  programs  of 
stimuli  on  which  the  trainee  should  receive  practice.  It  is 
the  synthetic  representation  of  programs  of  target  and  cursor 
relationships  which  can  run  into  major  technical  problems. 
These  problems  cannot  be  resolved  by  fiats;  they  can  at  best, 
however,  be  guided  by  behavioral  principals.  For  this  reason! 
the  relatively  concrete  and  torse  design  recommendations  ear- 
ly in  the  chapter  have  given  way  to  more  discursive  recommen- 
dations. 
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CHAPTER  IV.  KNOWLEDGE  OF  RESULTS  AND  SCORING 


Distinction  Between  Equipment  For  Training  vo Evaluation1 

Trainers  and  simulators  are  designed  and  used  both  for  training  and  for 
evaluation  of  trainees.  In  evaluating  trainees  it  is  important  to  have  re- 
liability and  standardization  in  the  presentation  of  test  exercises  so  that  each 
trainee  will  be  tested  and  evaluated  under  identical  or  highly  comparable  con- 
ditions, Ihis  requirement  is  not  imposed  upon  a device  used  only  for  training 

purposes.  That  is,  we  want  to  give  the  gunnery  trainee  information  or  cues  a- 
bout  his  sighting  errors  which  he  can  use  to  improve  himself j we  are  not  interest- 
ed in  detennining  whether  or  not  the  beginning  gunnery  trainee  has  enough  skill  to 
enter  combat.  Our  concern  in  this  chapter  will  be  principally  with  knowledge  of 
results  and  scoring  problems  as  they  deal  with  the  trainee's  learning  an 
operational  skill* 

Summary  Functions  of  Knowledge  of  Results  In  Training 

Broadly  speaking  we  may  think  of  knowledge  of  results  as  any  events  follow- 
ing a response  which  the  trainee  regards  as  a consequence  to  that  response.  Such 
events  may  range  from  "being  washed  out  of  training"  to  a "change  in  a pointer 
reading".  Knowledge  of  results  may  take  the  form  of  errors  in  estimated  time  of 
arrival  for  a navigator-bombardier,  time  on  target  for  the  gunner,  a lock-on  indi- 
cation for  the  radar  observer,  or  safe  landing  for  the  bomber  pilot.  Obviously  we 
will  have  to  have  a more  specific  set  of  terms.  These  new  terms  might  as  well  be 
rooted  in  the  several  ways  that  a trainee  can  use  knowledge  of  results  in  training. 

Motivation  to  learn;  For  the  time  being,  we  will  have  to  assume  that  the 
trainee  wants"  to" do  the  job  for  which  he  has  entered  training,  But  suitable 
indoctrination  must  also  induce  him  to  want  to  learn  the  tasks  in  the  job.  There 
is  a difference;  a trainee  who  is  motivated  only  to  perform  the  job  may  be 
exasperated  with  slow  practice,  although  it  may  be  true  that  only  through  slow  or 
part-task  practice  can  he  acquire  and  improve  his  skill. 

Presuming  that  the  trainee  is  motivated  to  learn,  knowledge  of  results  may 
provide  motive-incentive  conditions  by  acquainting  him  with  his  improvement  in 
the  course  of  practice.  Reinforced  by  evidence  of  improvement,  he  will  (other 
things  being  equal)  want  to  continue  practice. 

"he  trainee  usually  must  be  assisted  in  interpreting  gains  (and  temporary- 
losses)  in  improvement  especially  in  complex  skills  so  that  he  does  not  become  dis- 
couraged by  apparently  slow  progress,,  This  matter  is  treated  below  on  page  IV- 18, 

Reward  need  not  be  delivered  on  every  trial  in  order  that  the  trainee  will 
remain  motivated,  (Money  rewards  are  only  paid  employees  weekly  or  monthly.) 
Probably  each  practice  session  will  motivate  the  trainee  to  return  more  eagerly 
to  the  next  session  if  he  concludes  with  some  "feeling  of  achievement"  which  is 
rewarding,.  The  human  being  is  capable  of  responding  to  delayed  reward,  and 


1,  See  also  Chapter  VI,  Proficiency  Measurement, 
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many  symbolic  rewards  and  interim  reward  substitutes. 

The  problem  of  motivation  and  specific  incentives  to  learn  are  of  great 
importance  to  training  and  to  the  kinds  of  trainer  which  will  be  acceptable 
to  instructor  find  trainee.  The  problem  is,  however,  outside  our  scope  here. 

Orient  to  what  next  to  learn?  Knowledge  of  results  may  also  serve  to 
direct  the  trainee's  attention  to  some  major  part  of  his  output  which  requires 
special  concentration  of  effort.  Thus  the  bombardier  may  be  informed  that  his 
borab-run  procedures  are  unreliably  performed  whereas  his  tracking  is  at  least 
adequate,  or  the  student  pilot  may  be  proficient  in  stick  control  but  may  be 
lax  in  fuel  management  or  throttle  control.  The  automobile  driver  may  be  told 
that  he  is  proficient  in  steering  through  traffic  but  needs  special  practice  in 
parking  the  car.  One  of  the  features  of  complex  tasks  is  that  they  may  need 
to  be  learned  with  special  attention  devoted  to  certain  aspects  of  them  at  a 
time.  Trainee  may  need  to  shift  intensive  effort  from  one  aspect  to  another  in 
various  stages  of  practice.  Knowledge  of  results  may  provide  cues  to  when 
those  shifts  should  be  made. 

Information  as  to  what  next  adjustment  to  make  ("action  feedback11);  Knowl- 
edge of  results  may  serve  two  important  short-range  functions,  and  the  distinct- 
ion between  these  functions  is  important: 

(a)  The  consequence  of  a response  may  guide  the  nert  response. 

When  it  does,  we  may  refer  to  it  as  feedback  which  guides  performance, 
or  briefly  as  "action  feedback."  We  have  action  feedback  in  continu- 
ous tasks  and  in  discontinuous  tasks.  The  moment-by-moment  aiming 
corrections  are  guided  by  what  we  are  calling  action  feedback:  when 
light  A in  response  to  switch  A informs  the  operator  that  he  now 
3hould  press  switch  B,  light  A is  serving  as  action  feedback  in  a dis- 
continuous task.  Action  feedback  is  a cue  used  to  tell  what  to  do 
nexrt  in  a response  sequence. 


The  situation  is  diagrammed  as  follows: 


Stimulus  A 


Response  A 


Feedback 

stimulus 


Response  B 


rr\ 

(Press  switch  A)  Signal  lamp  A 
Used  as  cue-  to 


(Press  switch  B) 


(b)  Information  as  to  what  adjustment  should  have  been  made  ("learning  feed- 
laack'O:'  Response  feedback,  or  error  information,  may  also  be  used  byT.He 
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operator  to  tell  him  what  response  he  should  have  triad o (or  should  not  havo 
made)  to  some  previous  stimulus. 

A pilot  como3  in  for  a landing.  He  undershoots  the  runway. 

The  crash  informs  the  pilot  that  ho  should  have  made  a dif- 
ferent response.  If  the  pilot  survives  with  this  information, 
and  uses  it  so  that  he  does  not  commit  the  same  error  (in 
kind  or  degree)  we  may  refer  to  it  as  "learning  feedback." 


Learning  feedback  is  information  used  to  change  a response  or  habit  when 
the  same  or  similar  stimulus  recurs  in  the  future.  It  may  also  strengthen  a 
stimulus-response  pattern,  as  when  the  trainee  observes  an  indication  or  re- 
sponse correctness  and  says  to  himself,  "I  see  that  I did  what  I was  supposed 
to  do." 


Diagram  the  situation  like  this: 

Stimulus  A Response  A 


Feedback 

stimulus 


wrong  direction) 


V 


■v' 


"To  this  signal  I should  have  pressed  down." 


Learning  feedback  (as  we  have  defined  it)  helps  cut  down  the  trial-and- 
error  process  of  learning.  It  makes  explicit  the  fact  that  symbolic  activity 
"in  a person's  head"  can  guide  motor  responses  in  the  learning  process.  But 
it  also  suggests  certain  limitations  which  training  should  take  into  account. 
These  limitations  will  be  the  subject  of  a number  of  recommendations. 


The  same  signal  may  serve  as  learning  feedback  or  action  feedback,  or 
both,  depending  on  the  way  it  is  used  by  the  trainee. 

It  is  axiomatic  that  in  order  for  a response  to  be  learned  to  a given 
stimulus  or  situation,  it  has  to  be  made  to  that  stimulus.  That  is,  the  pilot 
of  conventional  jet  aircraft  must  make  the  response  of  reducing  control  stick 
forces  to  the  cue  of  aircraft  buffo'  ng  in  order  to  learn  how  to  avoid  stalls 
or  to  recover  from  stalls.  Many  young  pilots  have  experienced  difficulty- 
after  thorough  study  in  classroom  and  text  book.  Human  beings  have  the  capa- 
city to  make  symbolic  response  to  symbolic  (and  actual)  stimuli,  and  in  some 
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cases,  are  able  to  generalize  the  symbolic  S-R  into  an  actual  S-R.  Thi3  mat- 
ter was  presented  in  Chapter  I.  In  order  for  such  transfer  to  be  made,  how- 
ever, there  must  be  some  kind  of  one-to-one  relationship  between  the  symbolic 
stimulus  and  environmental  stimulus,  and  between  the  symbolic  response  and  the 
motor  response.  Thus  one  can  memorize  the  command,  "When  it  rains  I will  shut 
the  door."  But  if  the  person  is  unable  to  identify  a door  (stimulus)  or  is 
unable  to  perform  the  responses  denoted  by  "shutting  a door,"  the  act  will  not 
bo  performed. 

The  same  principle  holds  for  various  degrees  of  specificity  of  stimuli 
and  responses. 

Bearing  these  principles  in  mind,  a number  of  requirements  can  bo  set  down 
about  the  effective  use  of  learning  feedback. 


Conditions  Necessary  For  Learning  Feedback  To  Be  Effective 

To  the  extent  that  knowledge  of  results  is  to  be  effective  in  "voluntary" 
learning,  the  following  conditions  must  be  set: 

1.  The  trainee  must  conceptualize  the  critical  or  inciting  stimulus  in 
the  task.  Call  the  idea  cf  this  particular  task  stimulus  the  (lsym- 
bo?  Lc  stimulus." 


Associative  link 


Actual  unlit  lamp 


0 


"Idea"  of  unlit  lamp 
(symbolic  stimulus) 


2.  The  trainee  must  conceptualize  the  response  he  did  make  to  that 
stimulus.  Call  the  idea  of  this  response  the  "symbolic  response." 


switch  down 


Idea  of 

finger  pressing  switch  down 
(symbolic  response) 
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3.  The  trainee  mu3t  conceptualize  the  relation  between  the 
stimulus  and  the  response 


h. 


symbolic 

response 


The  trainee  must  conceptualize  a response  correction  and 
have  the  capacity  to  execute  his  idea  of  a correction. 


5.  The  trainee  must  relate  the  feedback  signal  to  tho  relevant 
stimulus  and  response  which  produced  the  feedback,  and  do 
so  the  next  time  the  stimulus  occurs. 


The  sequence  of  events  leading  to  response  corrections  can  be  shown  as 
follows  1 


(1) 


TRIAL  1 Actual 
Stimulus 


(2) 


incorrect 
— actual 
response 


(3) 


-feedback 


(li) 

(5) 

/ > 

f \ 

f symbolic 

' symbolic 

1 stimulus 

>< 

stimulus 

symbolic 

symbolic 

response 

response 

s(  incorrect), 

( correct) 

(6) 


(7) 


(8) 


TRIAL  2 Actual 
: " stimulus 


incorrect  response  tendency] 
symbolic  response  (correct) 


probability  of 
“correct  response 


This  diagram  illustrates  the  mechanism  whereby,  through 
voluntary,  symbolic  activity,  a correct  response  can  be  sub- 
stituted for  an  incorrect  response.  Repetition  of,  the  correct 
response,  especially  under  a variety  of  conditions,  increases 
its  probability  and  reliability.  Decreasing  ■the  time  limits 
to  several  seconds  between  events  2,  3,  b,  5 and  6 should  increase 
the  probability  of  the  correct  response  occurring . A very  short 
time  interval  between  6 and  7 may  increase  difficulty  of  substitut- 
ing the  correct  response  by  not  allowing  the  trainee  time  to  nthink 
out"  what  he  should  do  and  inhibit  the  old  incorrect  response. 

The  above  discussion  generates  a number  of  recommendations  for  train- 
ing equipment  design: 
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TRAINER  RECOMMENDATIONS 

* 1-  Dolay  between  response  and  learning  feedback:  The  shorter  the  time 

of  interval  between  response  and  learning  feedback,  the  better  for  efficient 
learning.  For  tracking  tasks  as  in  fighter  gunnery  this  best  minimum  inter- 
val between  pointing  of  the  aircraft  and  information  as  to  accuracy  of  track- 
ing is  0 seconds  but  for  discontinuous  tasks,  e.g.,  turning  on  and  checking 
radar  navigational  equipment,  there  is  suggestive  evidence  to  indicate  that 
about  l/2  second  delay  is  optimal. 

Important  notes  It  is  both  pointless  and  undesirable  to  simu- 
late operational  delay  between  response  and  feedback  if  the  trainee 
can  make  no  further  corrective  reapers e in  that  cycle  of  work.  Thus 
some  bombing  trainers  simulate  time  of  fall  of  the  bombs  before  hit 
indication  is  presented.  Since  after  bomb  release  the  bombardier 
can  do  absolutely  nothing  about  where  the  bombs  fall,  it  would  be  de- 
sirable to  present  him  with  immediate  hit  information  following  simu- 
lated bomb  release. 


2.  Delay  between  Incorrect  response  and  opportunity  to  correct  the  re- 
sponses: Depending  on  get-ready  periods  ( foreperiods)'  for  performing  a stimu- 
lus-Response,  the  shorter  the  time  between  an  error  trial  and  a corrective 
trial,  the  more  likely  the  corrective  response  oendency  will  occur.  The  train- 
ing equipment  should  be  able  to  immediately  repeat  a certain  trial  (with  its 
peculiar  target  track)  so  the  trainee  can  unlearn  the  inappropriate  responses 
to  that  series  of  stimuliand  practice  the  appropriate  responses.  For  example, 
when  a navigator  makes  a tracking  error  during  a wind  run,  the  training 
"flight"  should  be  stopped,  the  error  pointed  out  and  the  wind  run  repeated 
immediately  thereafter. 

3.  Recalling  the  critical  stimulus:  The  more  completely  the  trainee  can 
recall  the  stimulus  to  which  he  responded  erroneously,  the  more  likely  he  will 
reinstate  corrective  response  tendencies.  As  a practical  suggestion,  the 
training  situation  (device  or  instructor)  may  sometimes  warn  the  trainee  that 
the  critical  stimulus  is  about  to  appear.  The  artifical  warning  should  be 
discontinued  within  a trial  or  two  after  the  trainee  has  responded  correctly. 

1|.  Symbolizing  the  stimulus?  When  presenting  feedback,  the  more  the 
critical  task  stimulus  is  symbolized  rather  than  acuual,  the  less  likely 
there  will  be  functional* or  effective  recall  of  the  actual  task  stimulus. 

This  is  one  major  limitation  of  a graphic  or  permament  record.  On  the  other 
hand ,'Tf-sui table  reaction  time  is  allbwedTor  symbolic  associations  to  oper- 
ate, the  principle  may  have  less  practical  value  than  is  afforded  by  the 
flexibility  of  using  symbols.  Debriefing  sessions  do  provide  valuable  train- 
ing, even  though  the  words  are  only  symbolic  of  actual  events  occuring  in 
maneuvers. 
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Pre-training  In  The  Use  Of  Learning  Feedback 

Symbolizing  stimulus,  response  and  feedback  signals t Verbal  pre- 
training (or  other  symbolic  equivalents)  of  specific  stimuli,  responses 
and  response  feedback  signals  will  increase  the  ability  of  the  trainee 
to  profit  from  "learning  feedback."  ThiB  is  especially  true  if  there 
is  substantial  delay  between  the  S-R  in  ohe  task  and  the  presentation 
of  the  feedback. 

Identifying  sources  of  indication  of  response  adequacy?  The  trainee 
will  profit  from  pre-training  in  looking  for  and  identifying  sources  of 
cues  which  can  3erve  him  as  learning  feedback  — that  is,  as  indicators 
of  response  adequacy. 


Identifying  An  Error  Signal 
TRAINER  RECOMMENDATIONS 

1.  Ease  of  identifying  the  signal:  The  more  readily  and  clearly  the 
trainee  can  identify  a correct  from  an  incorrect  response,  or  an  on-target 
from  an  off-target  situation,  the  more  rapidly  can  learning  of  a task  or  sub- 
task occur. 

2.  Distinction  between  feedback  signals?  The  more  readily  the  trainee 
can  distinguish  feedback  signals  arising  from  one  set  or  mode  of  responses 
from  feedback  signals  arising  from  other  sets  or  modes  of  response,  the  more 
rapidly  he  can  learn.  It  would  be  easier  to  improve  dive  bombing  performance 
if  the  trainee  were  told  that  he  released  his  bomb  at  too  long  a slant  range 
or  did  not  make  sufficient  lead  allowance  than  if  he  were  simply  told  that  his 
bomb  fell  short. 

In  many  training  situations  the  instructor  assists  the  trainee  to  identify 
and  distinguish  among  feedback  signals.  Training  equipment  design  may  help  the 
instructor  by  providing  artificial  enhancement  of  specific  learning  feedback 
cues  to  the  instructor  and  to  the  trainee  (providing  the  latter  will  not  use  it 
as  an  "action  feedback)."  (See  page  72 0) 

3.  Strength  and  duration  of  learning  feedback  cues:  Learning  feedback 
cues  presented  early  in  learning  should  be  of  sufficient  strength  (signal-to- 
noise  ratio)  and  duration  to  permit  the  trainee  to  interpret  the  signal  and 
relate  it  to  the  appropriate  S-R. 

The  equipment  supports  for  this  type  of  practice  may  need  to  involve  no 
more  than  appropriate  Ow/OFF  switches  in  the  instructor^  station.  For  a con- 
tinuous task  a FREEZE  switch  can  allow  the  instructor  to  stop  the  current  pro- 
gram to  the  trainee  at  any  time.  This  device  has  already  been  used  in  gunnery 
and  navigation. 
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b*  Precision  of  feedback  information:  Tolerable  or  scorable  error  should 
not  impose  a ceiling  on  what  the  trainoe  can  attempt  to  do  in  eliminating  hie 
error.  Here  hit  vs.  no-hit  information  is  usually  too  crude  to  provide  maximum 
useful  information  to  the  trainee.  And  even  though  hit  scores  may  be  in  terms 
of  a 5 Mil  target,  the  trainee  should  be  able  to  try  for  smaller  error  by  being 
shown  the  position  of  his  his  hit  within  the  target  perimeter  unless  doing  so  is 
hostile  to  some  other  training  purpose. 

Self-perception  of  error:  The  trainee  should  be  encouraged  as  early 
as  possible  in  training  to  identify  error  signals  by  himself  for  learn- 
ing purposes.  For  example,  the  fighter  should  learn  to  recognize  an  ex- 
cessive G-suit  force  as  an  indication  that  he  is  about  to  exceed  G force 
limits.  In  this  way  he  can  maximize  the  usefulness  of  practice  both  in 
synthetic  and  operational  performance. 

Especially  during  the  early  stages  of  practice,  the  trainee's  ability 
to  learn  to  recognize  error  signals  Is  inversely  related  to  task  pacing 
imposed  by  the  instructor  or  the  equipment.  Self-pacing  of  tasks  has  some 
pay-off  early  in  learning  to  recognize  error  signals. 

5.  Feedback  pointing  to  stimulus  component  of  task*  In  general,  the  in- 
stractional  situation  should  direct  the  trainee's  attention  primarily  to  the 
stimulus  (or  the  mediating  concept)  and  secondarily  towards  the  critical 
response  as  such. 

Example:  The  instructor  telling  xne  trainee  "When  you 
have  passed  a car  and  before  you  turn  back  into  the  right 
lane,  (stimulus^  be  sure  to  look  in  the  rear  view  mirror 
to  see  that  you  have  passed  the  car.'1 

TRAINER  AND  INSTRUCTOR  RECCMMMMTIONS 

6.  Damping  excessive  response  oscillation s In  early  stages  of  learning 
a tracking  task  the  trainee  should  be  encooraged  to  disregard  minor  deviations 
or  errors  if  his  attempts  to  correct  for  them  lead  to  over -corrections  and  os- 
cillation of  machine  output.  It  may  actually  be  desirable  to  introduce  seme 
artificial  damping  effect  into  the  action  feedback  presented  to  the  trainee 
during  early  practice.  This  damping  should  be  removed  by  middle  and  late  stages 
of  learning. 

Example:  The  continuous  jiggling  of  the  steering  wheel 
by  the  : novice  driver. 

As  a corollary,  in  continuous  tasks  the  instructor  should,  where  safety 
permits,  allow  time  for  the  trainee's  error  to  become  large  enough  to  be  readily 
perceived,  preferably  without  calling  the  trainee's  attention  to  it. 
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Verbal  instruction  about  error  signals i Verbal  instructions 
may  be  ai  least  temporarily  effective  in  aiding  the  trainee  to 
learn  to  identify  error  or  feedback  signals. 

Example:  The  instructor  telling  the  engine  mechanic, 

"As  you  are  adjusting  the  carburetor  watch  to  see  if 
the  exhaust  is  a light  blue." 

Using  Feedback  Information  . 

GENERAL  RECCtMENEtATIONS 


1.  Interpretabillty  of  feedback  cues:  It  is  useless  for  learning  pur- 
poses to  present  the  trainee  with  learning  feedback  from  which  he  cannot  in- 
terpret what  the  correct  response  should  have  boon.  That  is,  the  trainee 
must  be  able  readily  to  infer  from  the  feedback  signal  (which  may  include  the 
instructor's  remarks)  what  kind  of  response  correction  should  be  made  in  fu- 
ture trials. 

2.  Matching  specificity  of  feedback  with  stage  of  learning:  The  amount 
of  learning  feedback  which  can  be  perceived  and  remembered  per  unit  of  per- 
formance will  obviously  differ  from  one  stage  of  learning  a complex  task  to 
another.  Rather  than  confuse  the  trainee  by  giving  him  more  information  than 
he  can  use,  it  is  better  to  allow  tho  equivalent  of  part- task  practice  in 
given  blocks  of  trials,  concentrating  feedback  information  on: 

a.  His  systematic  rather  than  occasional  errors 

b.  Response  systems  which  are  retarded  compared 
to  his  other  aspects  of  response  in  the  task 
or  job 

3.  Overburdening  the  trainee  with  excessive  learning  feedback:  To  the 
extent  that  the  trainee  cannot  put  into  effect  in  later  trials  Idle  learning 
feedback  information  presented  to  him,  the  excess  feedback  acts  as  noise  in- 
terfering with  perception  and  later  recall  of  such  information  as  he  might 
have  used.  Practice  and  learning  feedback  should  therefore  consist  of  those 
segments  which,  for  a given  stage  of  learning,  maximize  the  trainee's  making 
of  correct  response  when  he  is  performing  at  the  limit  of  his  momentary  capa- 
bilities. This  requirement  often  means  simpler  starting  exercises  than  is 
common  in  training,  and  more  frequent  rehearsal  of  such  simpler  exercises,  but 
with  shifts  of  emphasis  among  blocks  of  trials  from  one  performance  variable 
to  another.  For  example,  the  pilot  who  is  transitioning  to  jot  bombers  may  be 
taught  check  procedures,  flight  control  during  takeoff,  and  throttle  control 
to  achieve  masiraum  performance  separately.  The  nomenclature,  operating  prin- 
ciples, and  manipulatory  skills  would  be  very  difficult  to  learn  if  equal  em- 
phasis were  given  to  all  three  areas  at  one  time. 
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Trainee's  verbalization  of  correct  stimulus -res  pons as:  The 
training  situation  should  encourage  the  trainee  to  verbalize 
what  correct  response  he  should  have  made  to  the  task  stimulus 
when  he  was  in  error,  and  the  correct  response  he  did  make  to 
the  stimulus  when  he  was  successful.  Such  verbalization  should 
be  monitored  (as  by  the  instructor).  By  verbalizing  aloud  it 
can  be  monitored,  and  will  be  better  remembered,  at  leaBt  as  a 
verbalization.  This  principle  emphasizes  the  importance  of 
good  facilities  for  voice  communication  between  the  trainee  and 
his  instructor. 


TRAINER  RECOMMENDATION 

1.  Importance  of  eliminating  directional  error  in  performance!  Be- 
cause of  its  all-or-none  character,  the  elimination  o£  directional  error  in 
a response  will  generally  be  more  difficult  than  learning  to  change  other 
response  components  such  as  magnitude  and  rate.  This  is  especially  true  if 
the  response  is  one  of  a response-cued  series.  Special  attention  should  be 
given  to  inform  the  trainee  quickly  of  direction  errors  such  as  if  he  flipped 
a switch  in  the  wrong  direction  at  a given  time. 

The  importance  of  this  consideration  trifles  when 
a trainer  is  being  designed  to  transition  from  one 
aircraft  to  another.  Suppose  the  in-out  positions 
of  the  speed  brake  switches  of  two  airplanes  were 
reversed.  On  entering  certain  steep  dives  failure 
of  the  brakes  to  extend  might  well  be  fatal. 

2.  Occasional  sampling  of  total  task  performance:  The  programming  of 

practice  and  the  presentation  of  knowledge  of  results  information  should  be 
designed  to  search  throughout  the  trainee's  entire  task  repertory  at  frequent 
intervals  during  part- task  practice.  This  search  should  permit  the  discovery 
and  elimination  of  incorrect  response  tendencies  before  they  become  too  strong- 
ly learned  and  generalized.  , 

3.  Occasional  self -pacing  of  task  by  trainee:  The  training  situation 
should  provide  for  occasional  practice  trials  in  which  the  trainee  can  set  his 
own  pace  for  performance.'  Discontinuous  tasks  should  profit  more  from  such 
self-paced  trial3  than  continuous  tasks.  The  trainee  should  have,  on  such 
trials , ample  time  to  recall  and  verbalize  what  he  is  going  tc  do  next  from 
one  step  to  another. 

h.  Shifuing  from  part- task  to  total-task  context:  If  a part- task 
trainer  is  used  to  correct  habit  systems,  practice  on  the  part- task  should 
frequently  alternate  with  practice  of  that  part-task  in  the  total  task  con- 
text. Thi3  will  tend  to  guarantee  that  what  is  being  learned  in  part-task 
practice  will  transfer  to  the  total  task  context.  Practice  of  corrected  re- 
sponse should  always  be  made  to  a variety  of  situations  before  it  can  be  con- 
sidered reliable. 
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$•  Permitting  the  trainee  to  review  his  responses  in  task  contexts  Two 
general  methods  may  be  used  in  training  equipment  design  to  help  the  trainee 
focus  attention  on  his  performance  with  a view  to  the  specific  operations  where- 
by he  can  improve  it. 

Sufficient  time  may  be  allowed  between  cycles  of  practice  so  that  the  in- 
structor may  emphasize  the  review  of  successful  vs.  unsuccessful  components  of 
tack  responses  and  the  cues  to  which  they  were  made.  This  period  of  review 
should  probably  follow  each  cycle  when  it  is  longer  than,  soy,  a minute,  but 
after  alternate  cycles  when  the  task  sequence  is  shorter.  If  possible,  breaks 
should  not  come  when  the  trainee  has  task-directed  anticipations,  such  ns  mid- 
way in  a simulated  aerial  attack  pattern. 

If  it  is  necessary,  however,  to  break  up  an  undesirable  habit  pattern 
which  is  becoming  stabilized  within  a practice  cycle  the  instructc:  should  have 
a FREEZE  switch  which  renders  all  displays  and  controls  static.  Thus  the  in- 
structor may  interrupt  a performance  in  order  to  demonstrate  some  specific  in- 
adequacy which  may  be  difficult  or  impossible  to  show  at  the  end  of  the  cycle, 
or  be  unconvincing  to  the  trainee. 

The  instructor  should  be  cautioned  to  use  the  FREEZE  control  rarely  and  ju- 
dicious ly,  or  the  highly  motivated  trainee  may  become  disturbed  throughout  prac- 
tice. 


Operational  And  Artificial  Feedback 

An  important  distinction  should  be  made  between  feedback  information  which 
is  in  training  substantially  like  that  in  operations,  as  against  feedback  pro- 
vided in  training  which  is  characteristically  absent  in  operations.  The  latter 
may  be  thought  of  as  artificial  feedback  and  the  former  as  operational  feedback. 

Examples  of  artificial  feedback  include  the  comments  made  by  the  instruct- 
or about  the  trainee's  performance,  or  some  special  signal  not  normally  in 
operations  which  indicates  to  the  trainee  when  he  is  on-target  or  off-target. 

Artificial  feedback  may  often  function  either  as  action  feedback  or  as 
learning  feedback.  When  the  instructor  tells  the  trainee,  "Now  is  the  right 
time  to  put  in  trim,"  the  instructor  is  presenting  artificial  action  feedback 
from  the  trainee's  previous  response.  But  when  the  instructor  tells  the  trainee, 
"When  you  were  making  that  turn  you  should  have  increased  your  throttle  set- 
ting," the  instructor  presented  artificial  learning  feedback. 

These  distinctions  are  far  from  academic  as  we  will  see  in  some  of  the 
following  principles. 

Dominance  of  action  feedback  over  learning  feedbacks  Where  a single  art.i- 
ficial  or  operational  feedback  cue  can  simultaneously  act  as  both  learning  and 
action  feedback,  the  trainee  will  tend  to  respond  m*r©  to  the  action  feedback 
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aspect  of  the  cue.  A continuous  tracking  task  produces  feedback  to  which  the 
trainee  may  respond  either  as  action  feedback  or  learning  feedback  or  both. 

It  is  a challenge  to  the  inventive  ingenuity  of  the  designer  of  training  equip- 
ment to  devisr  a method  of  presenting  effective  learning  feedback  (as  such)  to 
the  trainee  which  does  not  violate  the  following  finding. 

Artificial  cues  combining  both  action  and  learning  feedback!  When  a 
trainee  has  become" dependent  on  an  artificial  cuo  as  an  action  cue  during  train- 
ing on  a continuous  task,  the  removal  of  that  cue  tends  to  deteriorate  perform- 
ance to  about  the  level  which  would  have  been  reached  by  similar  practice 
without  that  artificial  cue. 

Example : A nmbar  of  studies  in  gunnery  training  have 
provided  for  the  reddening  of  a target  when  the  trainee 
was  on- target.  The  reddening  of  the  target  was  an  arti- 
ficial feedback  cue.  When,  the  target  was  not  reddened, 
after  a practice  series,  the  performance  of  those 
trainees  dropped  back  to  about  that  of  the  control  group 
who  had  equal  amounts  of  practice  without  the  artificial 
(target-reddening)  cue. 

TRAINER  RECOMEfflATlON  * J 

I 

Although  thie  problem  is  not  yet  a closed  issue,  it  can  be  said  that  re- 
sults to  date  do  not  justify  introducing  artificial  cues  which  the  trainee 
can  use  as  an  action  cue,  that  is,  to  guide  his  next  response.  It  is  possible 
that  occasional  trials  with  such  artificial  cues  may  have  good  motivational  ef- 
fect. 

But  where  artificial  feedback  can  be  used  by  the  trainee  only  as  learning 
feedback  we  should  expect  improvement  in  trial-by- trial  performance,  other 
things  being  equal. 

4 

Thus  the  projection  of  target  path  and  cursor  path 
(hit  path)  immediately  after  a practice  attack  should 
be  beneficial.  How  beneficial  it  would  be  would  de- 
pend on  the  axtent  to  which  other  principles  cited  in 
this  section  may  have  had  their  requirements  fulfilled. 

Dominance  of  dramatic  cuesi  The  trainee  in  the  training  situation  will 
tend  to  respond  to  idle  most  dramatic,  easily  discriminable  and  (to  him)  most 
reliable  cues.  Stimuli  which  have  strong  contrast  to  their  background,  or 
which  are  sudden  in  onset,  or  move  rapidly,  will  tend  to  be  dominant.  Cues 
arising  from  actual  objects  (earth,  horizon,  apparent  size  of  trees  and  houses) 
will  tend  to  be  dominant  over  symbolic  cues  (artificial  horizon,  altimeter) 
unless  special  training  is  directed  towards  reversing  this  dominance.  (This 
point  is  of  particular  significance  in  planning  the  programs  of  contact  train- 
ers, or  visual  reconnaisance  trainers.) 
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TRAINER  RECCMMENMTIOHS 

Learning  to  disregard  misleading  cuest  If  the  trained  must  learn  to  dis- 
regard certain  dominant  or  habitual  but  misleading  cues,  it  is  necessary  to 
introduce  such  cues  in  frequent  training  exercises. 

Unreliability  of  cue  associations » If  two  feedback  cues  X and  I are 
equally  relevant  to  the  perf,drmance"o?  a training  task,  it  should  not  be  necess- 
arily assumed  that,  because  the  trainee  uses  one  of  these  cues  successfull,  he 
also  has  learned  to  use  the  other  cue. 

For  example,  night  driving  offers  at  least  two  sets  of 
cues  for  steering  a car,  the  center  line  and  the  right 
edge  of  the  road.  Both  cues  are  almost  always  present. 

Drivers  may  become  proficient  in  guiding  their  car  by 
one,  but  ho  uncertain  and  somewhat  erratic  when  forced 
to  depend  on  the  other.  The  availability  of  cues  in 
the  past  does  not  guarantee  that  responses  have  become 
associated  with  them. 

Learning  to  respond  to  several  types  of  feedback  cue:  If  the  trainee  must 
learn  to  be  able  to  respond  to  either  or  several  types  of  cues,  some  practice 
exercises  Bhould  omit  all  but  one  type  of  cuej  other  exercises  should  omit  all 
but  the  other  type  of  cuo. 

Reinforcement  of  partially  inadequate  modes  of  response i Responses  should 
not  be  rewarded  if  they  are  made  in  a mariner  which  will  interfere  with  higher 
degrees  of  learning  the  task.  (At  least,  practice  with  such  reward  should  quick- 
ly be  discontinued  when  the  response  component  being  trained  at  the  moment  is 
even  moderately  Well  learned. 

Thus  the  trainee  gunner  ( 'lexible)  may  be  rewarded 
with  high  hit  scores  on  a training  device  which  does 
not  penalize  erratic  movement.  The  trainee  can  get 
high  hit  scores  by  such  erratic  movement  as  would,  on 
the  operational  device,  produce  large  inaccuracies  in 
lead  angle  through  the  computing  mechanism.  High 
scores  3hould  indicate  and  reward  modes  of  response  which 
are  essential  to  good  criterion  performance,  not  modes 
of  response  which  are  contradictory  to  it,  or  some  aspect 
of  it. 


Spuriouf  Cues 

A spivious  cue  is  one  which  unintentionally  occurs  in  the  training  situ- 
ation but  is  not  in  the  operational  situation,  and  to  which  the  trainee  may 
make,  or  try  to  make,  systematic  responses.  For  example,  the  audible  click  of 
a relay  in  an  electric  chronograph  when  the  trainee  is  on  target,  or  the  reach- 
ing motion  of  the  instructor  visible  to  the  trainee  when  the  former  is  throwing 
a ■trouble”  switch,  may  guide  the  trainee  in  a way  not  appropriate  to  operations. 
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Sj  irious  cues  may  he  random  aB  wall  as  systematic  within  the  context  of 
the  tra  ning  task.  Aa  a hypothetical  example,  occasional  power  surges  mani- 
fest in  the  lights  or  general  Illumination  may  remind  the  trainee  to  scan  or 
perform  some  other  job  activity.  Although  such  fluctuations  may  be  random 
during  practice,  they  may  uniformly  elicit  scanning  responses  which  the  trainee 
might  imt  make  in  their  absence. 

In  some  circumstances,  often  not  predictable,  the  human  organism  is  highly 
ser  -itive  to  habit  formation  to  even  very  subtle  cues.  The  trainee  uho  is  mo- 
tivated to  "beat  the  training  situation"  will  be  especially  alert  to  such  cues 
and  try  to  fit  them  into  his  hypothesis  of  how  to  beat  the  machine  or  the  in- 
structs . 

TRAINER  RECCHMENDATION 

It  is  worth  considerable  effort  to  eliminate  both  random  and  systematic 
spurious  cues  perceptible  to  the  trainee  and  especially  spurious  cues  if  the 
trainee  can  (or  is  likely  to)  use  them  to  guide  his  next  responses  (that  is, 
as  action  feedback). 

Dramatizing  Learning  Feedback 

Although  artificial  cues  may  be  undesirable  if  the  trainee  can  use  them 
for  action  feedback,  special  cues  may  be  helpful  in.  emphasizing  learning  feed- 
back if  they  cannot  also  serve  as  a cue  for  action.  It  may  be  quite  desirable 
to  dramc  w.ize  a successful  bomb  hit,  or  a simulated  hit  on  an  enemy  aircraft  in 
synthed  i.  ginnery  practice,  or  any  other  termination  of  a task  cycle  in  which 
response  'ias  boen  highly  successful  — or  the  reverse.  On  some  flight  simu- 
lators tnere  is  the  sound  of  a huge  crash  and  harp  music  to  dramatize  the 
tra ’.nee’s  diving  the  aircraft  into  the  ground.  The  harp  music  may  be  of  doubt- 
ful worth  because  it  may  introduce  humor  into  what  should  be  a grim  situation, 
and  thus  distract  the  trainee  from  memorizing  the  cues  and  corrective  responses 
whuch  would  avoid  such  failures  in  future  practice. 

Finding  the  middle  road  between  overdramatizing  learning  feedback,  with 
its  risk  of  distracting  the  trainee  from  analysis  and  later  recall  of  error, 
and  the  often  feeble  sense  of  "closure"  or  task  completion  obtained  in  a syn- 
thetic training  situation,  must  be  left  to  the  individual  case.  Dramatic  feed- 
back wil probably  not  aid  learning  £er  £e,  but  it  may  have  motivational  im- 
port.-ace  in  bringing  the  trains  back  to  the  next  practice  session  or  cycle 
with  i.iterost.  This  may  be  especially  true  if  the  task  or  stage  of  learn- 
ing perm:  ts  relatively  small  ratios  of  success  to  failure. 

We  my  remember  that  to  adults  as  well  as  children  it  is  more  interesting 
in  target  practice  to  hit  a bottle  than  a piece  of  board,  and  a glowing  elec- 
tric light  bulb  than  a bottle. 

Scoring  . d Tolerance  Graduation 

A seo < ing  tolerance  defines  the  limit  between  a response  considered  a 
"faille"  tnd  one  considered  c.  "success."  Or  the  distinction  may  be  between 
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n "hit”  and  a ”mi83.B  Such  ocoreB  help  the  trainee  interpret  whether  he  is 
doing  satisfactory  or  unsatisfactory  work  in  a given  practice  cycle,  session 
or  part  of  a course.  Since  these  are  value  judgments,  they  are  in  the  area 
of  trainee  motivation. 

Notei  The  reader  is  reminded  of  this  distinction  we  are 
trying  to  make  between  ’’information  about  his  error  (in- 
formative feedback)”  and  an  evaluation  of  the  significance 
on  ’’success”  of  his  performance  which  we  call  a "Score.” 

A gunner  may  see  that  he  is  10  mils  from  target  point  with- 
out being  given  a score  of  success  or  failure. 


TlRAUffiR  RECOMMENDATIONS 

Ratio  of  hit-versus-miss  scoring: 

1.  In  order  to  keep  the  trainee  interested  in  trying  to  improve,  our  best 
bbt  seems  to  be  to  offer  him  about  a fifty-fifty  chance  of  getting  a success 
or  failure  in  any  trial  where  he  is  competing  with  himself.  Individual  differ- 
ences might  change  his  optimum  figure  which,  in  any  event,  is  only  a host 
guess. 


2.  If  the  trainee  is  one  of  a class,  it  may  be  more  realistic  to  select 
tllie  scoring  tolerance  so  that  fifty  percent  of  all  tries  in  the  class  during 

a given  stage  of  skill  are  successes  and  fifty  percent  are  misses  on  the  basis 
of  daily  scores.  It  would  certainly  be  undesirable  for  any  trainee  to  have 
either  practically  all  hits  or  all  misses  during  his  trial-by- trial  practice. 

3.  In  order  to  maintain  a more  or  less  constant  ratio  of  hit-miss  as 
skill  increases,  it  bocomes  necessary  to  make  scoring  tolerances  smaller.  The 
training  situation  should  therefore  be  flexible  in  this  respect. 

k.  It  should  be  strongly  emphasized  that  the  5$  chance  of  hit  scoring 
should  be  used  only  within  a practice  session  where  a number  of  trials  occur. 
Scores  at  the  end  of  the  practice  session  should  reflect  progress  towards  the 
criterion  as  described  in  the  next  paragraph. 


TRAINER  RECOMMENDATIONS 

Scores  reflecting  progress  towards  the  final  criterion? 

1.  Scores  denoting  progress  in  learning  will  primarily  have  motivational 
rather  than  informative  significance.  If  they  are  made  available,  they  should 
probably  be  shown  at  the  end  of  the  trainee’s  practice  for  the  day  or  at  least 
prior  to  his  next  practice  session.  Such  scores  ideally  would  show  not  only 
the  trainee’s  mean  performance^  but  also  his  variability  (reliability)  of 
performance. 
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2.  Scoring  units  must  be  fine  enough  so  as  to  reflect  improvement  over 
considerable  periods  of  practice;  on  the  other  hand,  they  must  not  be  in  units 
which  are  overly  sensitive  to  chance  fluctuations.  The  question  cannot  be 
solvod  purely  on  logical  grounds;  empirical  data  are  necessary. 

3*  It  is  also  helpful  to  identify  a final  criterion  for  performance. 

This  criterion  may  be  based  on  the  performance  of  top-level  operators  or 
trainee  graduates,  or  it  may  be  baaed  on  some  criterion  acceptable  to  the  user 
of  the  trainees  who  have  graduated.  A criterion  of  final  performance  anchors 
the  goal  of  practice. 

h>  Learning  curves  characteristically  show  rapid  improvement  in  earlier 
stages  of  practice,  and  smaller  increments  of  improvement  per  given  unit  of 
later  practice.  The  small  apparent  improvement  reflected  in  positions  on  the 
curve  in  later  practice  may  be  discouraging.  If  an  empirical  improvement 
curve  on  the  task  is  avail  able,  it  might  be  modified  bo  as  to  take  account  of 
the  increasing  difficulty  in  attaining  given  increments  of  improvement. 
Graphically  this  would  be  no  problem.  It  would  require  stretching  the  units 
on  the  ordinate  at  the  upper  end  so  as  to  change  the  empirical  negatively 
accelerated  curve  into  a straight  line.  Normative  data  on  training  the  task 
would  be  required  in  order  to  obtain  the  empirical  curve. 


Practice 


Practice 


Standard  Improvement  Curve 


Modified  Curve 


5>.  The  simplest  method  of  converting  performance  data  into  any  kind  of 
evaluational  or  normative  score  is  to  provide  the  instructor  with  appropriate 
conversion  tables. 

Excluding  Machine  Error  Fran  Informational  Feedback 

To  confound  information  about  which  the  trainee  can  take  corrective  action 
with  information  about  which  he  can  do  nothing  is  obviously  inefficient  for 
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learning  pttrpooes.  It  is  partly  because  a synthetic  situation  often  con 
eliminate  random  machine  error  inherent  in  operational  equipment  that  syn- 
thetic training  can  be  more  efficient  than  training  in  operations. 

For  example,  there  in  always  substantial  random  error  in  gunnery 
equipment.  When  the  operator  fires  a weapon  in  operations,  his 
own  aiming  error  is  confounded  with  the  gun  error  or  unreliability. 

To  include  the  gtm  error  in  knowledge  of  results  dilutes  the 
adequacy  with  which  the  trainee  can  possibly  take  corrective  action 
the  next  time  he  aims  the  gun.  Since  the  purpose  in  training  is  to 
teach  him  to  aim  the  gun  to  the  best  of  his  ability,  only  point-on- 
aim  information  should  be  returned  to  him. 

TRAINER  RECCMMENDATIONS 

It  should  be  emphasized  that  the  reccsunenaation  is  to  eliminate  random 
machine  from  informational  feedback  to  the  trainee.  Constant  or  consistent 
machine  error,  by  definition,  is  a kind  of  error  for  which  the  trainee  can 
and  should  learn  to  compensate. 

There  are  two  qualifying  conditions  to  this  assertion. 

1 1.  If  the  trainee  will  acquire  a false  expectation  of  success  in  opera- 

tions because  he  was  trained  without  random  machine  error,  the  trainee  should 
be  given  oither  practice  or  indoctrination  which  will  make  his  expectations 
realistic  in  operations.  This  practice  or  Indoctrination  should  occur  after 
he  has  reached  a high  degree  of  skill  in  the  synthetic  task,  since  we  do  not 
expect  the  situation  to  be  optimal  for  learning. 

2.  If,  following  training  on  a "perfect  gun"  or  other  equipment,  the 
trainee  erroneously  attempts  in  operations  to  compensate  for  machine  errors 
added  to  his  own  aiming  errors,  there  should  be  a transition  phaso  of  train- 
ing. This  transition  phase  should  follow  a high  level  of  skill  in  aiming 
the  "perfect  equipment."  Machine  error  at  least  roughly  comparable  to  that  of 
operational  devices  should  be  introduced  in  order  to  acquaint  him  with  the  peri- 
meter of  the  zone  of'  error  about  which  he  can  do  nothing  (and  should  do  nothing 
if  smoothness  of  tracking  is  important)  by  way  of  compensatory  response. 

Another  reason  for  introducing  some  practice  in  the  context  of  the  total 
man-machine  error  system  is  that  he  will  not  be  led  into  foolhardy  risks  on 
ona  hand,  or  become  unduly  cowed  or  timid  on  the  other.  We  should  not  expect 
great  gains  in  aiming  skill,  however,  after  this  introduction  of  machine  error 
if  the  machine  error  is  a sizeable  component  of  the  total  system  error. 

(For  a more  thorough  treatment  of  this  problem  see  Miller,  Robert  B., 
Memorandum  No.  1,  Operational  Equipment  Error  Systsas  in  the 
Training  Equipment,  29  July  1953/®  supplement  to  Handbook  on 
Training  auiPfcfjMng Equipment  Design.  Copies  of  this  memorandum 
are  available  from  the  Psychology  branch,  Aero-Medical  Laboratory, 

Wright  Air  Development  Center,  Dayton,  Ohio.) 
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Doternlning  What  Information  To  Feed  Back  To  The  Trainee 

(a)  List  the  task  variables  (discriminations,  decisions,  motor 
activities)  which  the  trainer  is  intended  to  train' 

Action  feedback t The  first  step  should  lead  to  the  task  analysis  of 
operations.  From  it  we  determine  what  action  feedback  in  operations 
guides  responses  one  after  another  within  a task  cycle  or  job  seg- 
ment. There  is  no  choice  but  to  represent  those  in  the  training 
situation  whether  to  seme  dogree  of  realism  as  they  occur  in  opera- 
tions, or  symbolically.  They  may  be  symbolized  in  the  form  of  a 
statement  by  the  instructor,  or  they  may  be  coded  into  same  display 
shown  by  the  training  equipment. 

Examples : A procedural  task  in  which  a number  of  switches 
must  b'o~tlirown  in  a standard  sequence.  At  each  throw  of  a 
switch  in  a mock-up  the  instructor  may  say  '’Correct"  or 
"Incorrect."  Or  a special  tel- light  in  the  mock-up  may 
inform  the  trainee  that  his  correct  throwing  of  a given 
switch  now  authorizes  his  throwing  the  next  switch  in  the 
task  series.  In  operations,  the  successful  order  of  throw- 
ing switches  results  in  the  feathering  of  a propeller  with- 
out engine  damage. 

As  noted  elsewhere,  action  feedback  may  also  serve  as  learning  feedback. 

Learning  feedback?  From  the  task  analysis  determine  those  discriminations  and 
motor  responses  in  the  operational  task  for  which  action  feedback  (paragraph 
above)  is  not  in  itself  sufficient  information  to  tell  the  trainee  whether  his 
previous  response  was  right  or  wrong,  acceptable  or  unacceptable.  That  is, 
a final  machine  output  may  be  the  joint  result  of  a response  pattern  or  se- 
quence. In  radar  bombing,  for  example,  the  final  machine  output  is  the  cir- 
cular error  of  the  bomb  hit.  But  the  circular  error  does  not  reveal  to  the 
trainee  which  of  his  several  procedural  or  continuous  activities  contributed 
to  the  deviation  from  the  target  center.  Analysis  of  trainee  response  error 
must  be  provided  in  this  case  either  by  instructor  or  artificially  by  the 
trainer. 

When  these  response  components  of  the  task  have  been  abstracted,  the  de- 
signer will  have  to  decide  whether  it  is  more  practicable  for  the  instructor 
to  be  the  error  observer  and  source  of  corrective  information,  or  whether 
the  training  equipment,  or  some  combination  of  instructor  plus  signalling 
method  is  most  practicable. 
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In  this  way,  tentative  provision  for  correction  of  error  is  made  for  all 
essential  response  requirements  in  the  task.  The  particular  way  in  which  the 
equipment  displays  the  information  to  the  trainee  is  a later  step  in  human 
engineering. 

(b)  Determine  the  performance  tolerances  for  the  stages  and 
phases  of  learning  through  which  the  trainer  is  intended 
to  carry  the  trainee. 

(c)  Determine  ( tentatively  if  necessary)  which  of  these  var- 
iables will  have  instrumental  assistance  to  the  instructor 
by  Hi's  Having  information  about  them  automatically  dis- 
criminated cod  el,  stored  or  cociainlcated. 

Concluding  Statements 

The  principal  concern  of  the  designer  of  training  equipment  about  feedback 
to  the  trainee  may  be  summarized  as  follows: 

1.  What  are  the  minimum  types  or  channels  of  "'learning 
feedback"  which  will  be  most  helpful  in  getting  the  trainee  to 
reduce  or  eliminate  his  errors? 

2.  How  can  "learning  feedback"  be  emphasized  and  present- 
ed to  the  trainee,  but  without  its  being  used  as  a crutch  for 
"action  feedback"? 

3.  Is  the  instructor  or  the  equipment  likely  to  be  the 
most  efficient  and  economical  means  of  (a)  noting  response  error 
and  (b)  informing  the  trainee  about  it  as  learning  feedback?  If 
the  instructor  is  to  be  used,  what  does  this  mean  to  the  design 
of  the  instructor's  station?  These  problems  must  consider  the 
total  load  on  the  instructor  during  the  time  he  may  have  to  notice 
and  communicate.  Obviously  the  use  of  the  instructor  offers  flexi- 
bility and  economy  (unless  more  instructors  are  thereby  required)  $ 
the  use  of  equipment  entails  greater  reliability  and  objectivity. 

U.  How  car.  information  be  presented  to  the  trainee  so  that 
he  can  best  use  it  to  improve  his  habit  systems  in  his  repeated 
practice  trials? 

And  finally,  it  is  important  to  distinguish  between  problems  which  center 
on  the  following:  (l)  feedback  to  the  trainee  which  enables  him  to  improve 
his  specific  response  patterns,  (2)  evaluation  of  the  trainee  with  respect  to 
norms  or  to  other  trainees,  and  (3)  scoring  considerations  involving  trainee 
motivation  to  learn.  The  problem  of  trainee  evaluation  should  not  obscure 
the  prime”" importance  of  how  to  get  the  trainee  to  learn  most  rapidly  and  with 
maximum  transfer  value. 
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The  adequacy  of  plans  for  handling  knowledge  of  results  may  be  deter- 
mined from  the  reply  to  the  following  question t 

"Can  the  trainee  quickly  and  meaningfully  find  out 
whether  any  crucial  responses  he  is  making  are  within 
or  outside  each  of  the  quality  standards  required  of 
the  tasks?" 

For  each  behavioral  step  or  task  variable  the  designer  should  be  able  to 
write  in  tT^oncrete  Indication  to  the  Trainee  of  Response  Adequacy  or 
Inadequacy. " 


CHAPTER  V.  RECORDING  PROCEDURES 


Distinction  Between  Teaching  And  Evaluations!  Purposes 

Recording  of  trainee  performance  provides  a "memory"  for  scores.  This 
’’memory"  may  consist  of  a permanent  or  semi-permanent  record.  The  recording 
of  scores  for  training  purposes  may  require  different  data  than  recording  for 
evaluations!  purposes. 


Recorded  Data  As  A Help  In  Teaching 

Recorded  data  may  serve  as  a direct  aid  to  the  trainee  or  primarily  as 
a direct  aid  to  the  instructor.  Eaoh  of  the  four  following  topics  briefly 
discuss  how  they  may  provide  such  resistance,  and  suggest  recommendations  for 
trainer  design. 

1.  Showing  constant  versus  variable  errors : The  comparison  of  records 
of  performance  over  a nunW  of  trials'  or  practice  sessions  over  similar  kinds 
of  exercise  may  reveal  that  the  trainee  makes  some  errors  quite  consistently, 
whereas  other  errors  are  not  consistent.  The  instructor  and  trainee  will 
usually  want  to  concentrate  on  those  habit  patterns  which  consistently  produce 
undesirable  results.  If  the  recording  permits  such  analysis  readily  to  be 
made,  it  provides  obvious  help  in  making  such  decisions. 

Example:  A student  gunner  has  a pattern  of  shots  in  success- 
ive passes  at  a target:  seventy  per  cent  of  his  shots  are  to 
the  left  of  the  target. 

2.  Showing  continuities  of  trainee  response:  The  response  outputs  of  the 
trainee  may  be  so  complex  that  an  analytical  record  may  be  necessary  for  ade- 
quate learning  feedback.  Complete  records  of  trainee  gunnery  attacks  may  pro- 
vide data  relating  to  each  other  error  in  azimuth,  elevation,  ranging  and 
triggering. 

The  limitation  of  such  records  for  learning  purposes  is  that  they  are 
presented  too  long  after  the  trainee  has  made  the  responses.  Another  dis- 
advantage is  that  the  response  pattern  is  not  readily  related  to  a stimulus 
pattern  to  the  trainee  in  such  a way  that  he  can  use  the  recorded  data  for 
habit  correction,  (pages  6h=66*) 

Ideally,  the  record  should  (l)  be  flashed  back  to  the  trainee  immediate- 
ly after  the  response  cycle  (such  as  an  attack);  (2)  show  the  stimulus  as  well 
as  the  response  track  with  time  indications  which  permit  matching  them;  and 
(3)  such  stimulus  cues  as  will  permit  the  trainee  to  jjet  prepared  to  avoid 
some  particular  error  when  that  practice  cycle  is  repeated! (Phosphorescence 
might  be  a suggested  solution  for  parts  one  and  two  of  this  problem.) 
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3.  Showing  relations  of  one  part  of  performance  to  another t The  memory 
systom  intrinsic  to  recording  systems  may  bo  used  to  show  the  trainee  how  in- 
adequacies of  response  in  one  phase  of  a job  segment  were  reflected  in  an  in- 
adequacy in  another  job  segment.  Thus  ho  may  see  from  a navigational  record 
of  his  flight  path  why  he  failed  to  rendezvous  at  the  proper  time  and  place, 
or  why  his  bomb  missed  the  target.  Loosely  speaking,  recorded  data  may  help 
the  trainee  learn  the  "tactics”  of  hie  job.  The  memory  of  the  trainee  and 
instructor  both  may  be  inadequate  or  unconvincing  for  such  a purpose. 

h*  Aiding  the  overloaded  instructor:  The  instructor  in  modern  training 
devices  is  often  heavily  loaded  both  in  sensory  and  motor  demands.  An  auto- 
matic recording  system  may  reduce  this  load.  It  may  not  always  be  necessary 
for  the  recorder  to  run  continuously;  sometimes  it  may  be  sufficient  to  sense 
and  store  data  for  no  more  than  a few  critical  seconds.  If  itB  continuous 
operation  requires  heavy  monitoring  from  the  instructor,  it  may  be  as  great  a 
liability  as  an  asset  — or  more  so. 

For  example,  the  instructor  in  a fighter  gunnery  trainer 
may  need  to  ‘fly1  the  target  and  monitor  flight  instruments  slaved 
to  the  trainee's  instruments.  A graphic  recording  of  the  flight 
track  would  provide  the  instructor  a record  on  the  adequacy  of 
flight  technique,  a record  of  value  which  could  not  be  captured  on 
a memory  basis  by  the  instructor.  Track  made  good  during  entrance 
into  the  attack  course  and  during  pullout  1b  critical.  On  the 
other  hand,  a continuous  record  from  beginning  to  end  of  right-left, 
forward-back  'stick'  movement  would  become  meaningless  detail. 

Recording  may  be  desirable  both  for  information  which  flows  very  rapidly 
according  to  human  capacities  to  receive  it  (such  as  a gunnery  pass)  and  for  in- 
formation which  flows  very  slowly  (such  as  in  some  navigational  practice  sessions.) 
The  procedures  which  provide  scoring  variables  and  scoring  equipment  may,  by 
logical  extension,  help  determine  what,  if  any,  recording  equipment  may  be  de- 
sirable for  the  instructor's  station. 

SPECIAL  RECOMMENDATIONS 

If  the  recording  method  produces  a graphic  record,  provisions 
may  be  made  for  the  instructor  to  scribble  notations  or  symbols 
which  will  help  him  in  later  recall  when  he  reviews  the  trainee's 
performance.  It  should  be  ea3y  for  the  instructor  to  add  such  no- 
tations. Even  if  it  is  impracticable  for  him  to  be  able  to  write 
on  the  tape,  some  signal  marker  might  be  provided  for  him  to  use 
with  a switch.  This  signal  marker  would  show  on  the  tape  the  point 
at  which  the  instructor  had  a comment  to  make,  and  thus  help  . emind 
him  of  the  coitment. 

Important  Considerations  In  Design  Of  Recording  Equipment  For  Teaching 

Preliminary  thought  to  the  addition  of  recording  equipment  to  the  training 
device  should  be  subjected  to  a number  of  realistic  questions.  The  answers  to 
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these  questions  may  lead  to  revision  or  abandonment  of  recording  for  training 
purposes;  although  it  may  be  possible  that  the  equipment  might  be  retained  for 
evaluational  purposes  alone. 

1.  What  demonstrable  contribution  does  a record  have  in  this  particular 
training  problem  over  transient  presentations  of  information  (that  is, 
information  which  is  erased  by  or  pooled  with  the  next  response  the 
trainee  makes)?  By  precisely  what  steps  will  the  instructor  (or  trainee) 
transform  the  recorded  data  into  bettor  task  performance?  Why  could  not 
precisely  the  same  objective  be  reached  with  greater  flexibility  without 
the  record? 

2.  How  much  time  is  required  of  the  instructor  to  decode  or  decipher  the 
recorded  data  into  information  which  the  trainee  can  use  to  guide  his 
specific  behaviors  in  the  task? 

3.  How  much  attention  is  required  of  the  instructor  to  monitor  the  record- 
ing equipment?  Might  not  this  attention  better  be  directed  towards 
transient  scores  or  by  watching  directly  the  trainee's  performance? 

1;.  Does  the  record  permit  accurate  matching  of  input  and  output  to  and 
from  the  trainee?  That  is,  can  the  instructor  help  the  trainee "match 
his  response  error  with  the  inciting  stimulus  so  that  the  trainee  may  get 
a mental  "picture"  of  how  and  when  to  make  proper  response  corrections  over 
his  previous  performance?  How  reliably  does  the  scoring  arrangement  per- 
form insofar  as  matching  inputs  and  outputs  of trainees? 

5.  Does  the  record  permit  direct,  unambiguous  comparison  of  what  the 
trainee  should  do  and  what  he  does  do? 

6.  For  how  many  phases  of  learning  intended  for  training  on  the  trainer 

is  the  form  of  recording  appropriate?  Thus  an  elaborate  recording  arrange- 
ment may  be  useful  for  no  more  than  a brief  portion  of  the  training  problem. 
If  behaviors  in  this  portion  of  training  are  critical  to  later  job  perfor- 
mance, or  are  made  more  difficult  to  learn  without  scoring  records,  the 
recording  equipment  may  be  justified,  in  spite  of  the  limited  time  during 
which  it  is  used. 

?•  Is  the  scoring  mechanism  dependable  in  operation?  Can  difficulties 
(such  as  failure  of  pen  or  stylus  to  write)  be  corrected  in  a few  seconds, 
or  are  standby  mechanism  provided  for  immediately  taking  over  an  impaired 
function? 

8e  What  provisions  for  stand  by  scoring  are  available  to  the  instructor 
if  the  recording  equipment  goes  out?  Sore  training  devices  have  been  so 
designed  that  they  are  completely  useless  for  training  if  the  recording 
equipment  fails.  The  instructor  should  have  alternate  modes  of  getting 
at  least  partially  adequate  data  for  training  purposes  while  the  recording 
equipment  is  under  maintenance  or  repair. 
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9.  Hhat  provisions  are  made  for  simple  calibration  and  adjustment? 

Recording  For  Kvaluational  Purposes^- 

The  assessment  of  either  relative  or  absolute  proficiency  places  a prem- 
ium on  objectivity  and  reliability  of  performance  measurement.  Objectivity 
implies  freedom  from  personal  or  other  bias  introduced  into  measurement  by  an 
observer  or  handler  of  the  performance  data.  Reliability  refers  to  the  extent 
that  conditions  which  produced  measurement  at  time  A are  reproduciblo  in  a 
measurement  to  be  made  at  time  B, 

Theoretically,  scoring  and  recording  should  be  just  as  objective  in  train- 
ing as  in  evaluating.  Practically,  however,  the  interactions  with  the  instruct- 
or characteristic  of  training  permit  ironing  out  of  some  unreliability,  espec- 
ially over  a number  of  practice  trials.  But  evaluation  ia  more  or  less  a one- 
shot  affair  in  which  unreliability  in  observation  or  recall  by  the  instructor 
lacks  opportunity  for  correction.  Hence  it  becomes  important  to  relieve  the 
instructor  as  much  as  is  practicable  of  matters  involving  attention  and  adjust- 
ment in  evaluation  and  proficiency  testing. 

The  problem  of  relating  stimulus  to  response  1b  not  crucial  in  records  used 
for  proficiency  evaluation,  as  it  is  for  training  purposes.  In  general,  the 
question  asked  of  the  data  is,  "Did  the  trainee  fulfill  the  requirements  (of  the 
job,  or  of  training  as  the  case  may  be)  or  did  he  not?"  For  strictly  administra- 
tive purposes  it  may  sometimes  be  helpful  to  know  which  requirements  were  or 
were  not  fulfilled j but  for  training  purposes  it  would  be  absolutely  essential 
to  know. 


Summary  of  Recording  Methods  And  Types  of  Devices 

Any  recording  device  will  include  the  following  features: 

1.  A data  pickup  or  sensing  member.  This  member  may  be  coupled 
either  directly  to  the  control  activated  by  the  person  trained 
or  tested,  or  it  may  be  coupled  to  some  machine  output  linked 
to  the  control  activated  by  that  person. 

2.  A signal  amplifier  or  encoder.  This  mechanism  translates  the 
picked  up  signal  into  values  or  units  which  are  comparable  from 
one  sample  performance  to  another. 

3.  The  inscriber.  This  is  the  instrument  which  leaves  a track  or 
index  marking  on  some  permanent  or  semi-permanent  material. 


1.  Discussion  related  to  Recording  for  Evaluation  can  be  found  in  Chapter  VI, 
Proficiency  Measurement. 
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li.  The  record.  The  material  upon  which  the  translated  job 
operation  is  fixed  as  a datura. 

Note  that  any  one  or  a combination  of  these  four  items  can  be  performed  by  a 
human  observer  with  a load  pencil  and  a piece  of  paper. 

The  classification  of  recording  devices  may  be  according  to  same  one  or 
more  of  these  portions  of  the  recording  syetan.  Thus,  optical  recorders  are 
identified  by  their  method  of  picking  up  and  "inscribing11  data  — presuming 
the  datura  consists  of  light  rays  impressed  on  a photographic  emulsion.  The 
following  list  is  not  exhaustive]  nor  does  each  item  include  all  four  factors 
of  sensing  mechanism,  recording  and  transmission  mechanism,  inscribing  pro- 
cedure and  record  material.  The  list  is  intended  to  be  suggestive  of  record- 
making possibilities. 

1.  Instructor  makes  noues  on  paper. 

2.  Instructor  makes  notes  on  a check- list.  This  procedure  is 
somewhat  more  objective  than  1. 

3.  Instructor  speaks  into  a tape-recorder.  He  may  be  guided  by 
a check-list.  Or  instructor  may  activate  switches  which  put 
code  marks  on  a moving-drum  recorder. 

li.  Continuous  time  accumulators  are  photographed,  or  instructor 
takes  record  by  hand.  Examples  of  time  accumulators  are 
chronoscopes,  Standard  Electric  Timers. 

5.  Counters  with  data  photographed  or  impressed  on  tape  at  given 
intervals.  Counters  will  accumulate  any  discrete  units,  in- 
cluding time  marked  into  pulses. 

6.  Moving  drum  recorders  (kymograph  type).  A continuous  strip 
moves  at  constant  speed  past  one  or  more  recording  styli.  The 
stylus  may  write  with  ink]  a thermoelectric  contact  may  burn 
the  paper]  or  mechanical  pressure  of  the  stylus  may  trace  lines 
on  a waxed  surface. 

7»  Moving  crab  recorders.  These  are  a variation  of  6,  except  that 
the  record  bed  remains  stationary  and  the  writing  platform 
moves  in  two  coordinates  guided  by  a selsyn  system.  Time  can  be 
indicated  by  a special  periodic  pulse  delivered  to  the  recording 
pen  or  tracer. 

8.  Optical  pick-ups  transform  light  into  a photographic  record 

either  on  stationary  emulsion,  continuously  moving  paper  or  film, 
or  in  discrete,  successive  frames. 
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9.  Phosphorescence  could  be  utilized  to  record,  at  least  during 
a brief  period,  the  track  of  special  light  beans  (infra-red). 

10.  Card  punches  (international  Business  Machine  type)  can  re- 
cord many  categories  of  simultaneous  discrete  data. 


Mechanical  Properties  of  Recording  Mechanisms!  Design  Considerations 

It  nay  be  only  partly  oppropriate  to  point  out  several  important  engineer- 
ing considerations  in  the  design  of  recording  equipment.  Since  these  con- 
siderations will  be  generally  well  known  to  engineers,  it  may  suffice  merely 
to  mention  them. 

1.  Sensitivity:  How  small  a hraan  response  is  matched  by  a correlated 
reponse  in  the  record,  appropriate  in  direction,  magnitude  and  time?  The 
required  limits  of  sensitivity  for  recording  can  be  determined  on  the  same  basis 
as  are  the  required  limits  in  scoring. 

a.  In  gdheral,  they  should  be  based  on  the  limits  of  what 
the  highly  skilled  operator  himself  can  or  need  do 
nothing  about  by  way  of  correction  either  as  "action 
feedback"  or  "learning  feedback." 

b.  The  second  appeal  should  be  to  systems  outputs.  Sensitivity 
in  recording  need  be  no  greater  than  to  differentiate 
tolerance  from  non- tolerance  performance  when  recording  is 
used  for  proficiency  measurement.  This  statement  assumes  use 
of  an  "absolute  criterion."  A good  general  rule,  in  any 
event,  is  +hat  recording  of  response  (in  magnitude  or  in 
kind)  which  makes  no  difference  to  the  system  output  is 
unnecessary. 

c.  If  recording  is  to  be  used  for  research  purposes  not  directly 
connected  with  training  or  proficiency  evaluation,  these  pur- 
poses will  have  to  dictate  the  degree  of  sensitivity  required 
of  the  recording  mechanism. 

2.  Recording  range:  Recording  range  should  be  based  upon  the  purposes 
intended  for  recording.  If  the  record  is  to  be  used  for  training,  the  range 
should  extend  somewhat  beyond  the  poorest  and  best  performances  when  the  trainee 
is  subjected  respectively  to  the  most  difficult  and  the  least  difficult  "pro- 
grams" or  exercises. 

If  the  record  is  to  be  used  f . ;•  evaluation,  the  necessary  range  will  de- 
pend on  the  method  of  evaluation,  whether  it  will  bs  pass -fail  with  respect  to 
an  absolute  criterion  of  performance  or  on  trainee-trainee  comparisons  and 
norms.  If  the  former,  the  range  need  not  extend  far  beyond  the  tolerance 
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limits  established  for  criterion  performance.  If  evaluation  is  to  bo  baaed  on 
group  norms  (which  it  usually  1b)  then  recording  range  will  have  to  be  baaed 
on  empirical  data  which  showi  dispersion  ( standard  deviation)  of  sample  testees 
on  the  given  task.  If  the  sample  of  testeos  is  repressntativo  of  the  popula- 
tion to  be  tested,  three  standard  deviations  on  each  side  of  the  mean  shold 
provide  a range  more  than  adequate  for  all  practical  purposes.  This  range 
would  accurately  locate,  with  respect  to  relative  position  in  performance,  all 
but  0.3  per  cent  of  the  popul  tion  to  be  tested. 

3*  Backlash  on  training  or  performance  of  Tasks?  Performance  on  the 
operational  or  training  task  anould  be  unaffected  by  the  process  of  recording. 
That  is,  recording  should  not  introduce  artifacts  into  either  the  displays  or 
control  systems  in  the  testae's  work  situation.  The  substitution  of  optical 
and  electronic  methods  for  mechanical  methods  of  data  pick-up  has  reduced  the 
problem  of  backlash  on  equipment  linkages. 

Recording  may  introduce  another  kind  of  backlash,  howerver.  Thus,  the  in- 
structor may  have  to  require  the  trainee  to  make  artificial  pauses  mid-task  in 
order  to  readjust  the  recording  mechanism,  or  change  a tape.  Or  sounds  and 
other  cues  may  provide  spurious  information  to  the  trainee  or  person  tested. 

U.  Reliability  of  Recording:  The  validity  of  any  score  is  almost  in- 
evitably attentuated  by  unrel ' ability  introduced  through  scoring  and  recording 
of  performance  data.  The  pra  deal  meaning  of  this  unreliability  is  that  test- 
ees will  be  passed  who  would  '^nd  should)  have  been  failed,  and  some  testees 
will  be  failed  who  should  have  oeen  passed. 

It  may  be  desirable  t-c  provide  some  standard  inputs  by  which  to  test  the  re- 
liability of  the  scoring  and  recording  mechanisms.  These  inputs  should  sample 
from  the  range  of  pxrogra  s and  response  outputs  of  the  tasks  on  which  behavioral 
outputs  are  being  measur  d.  The  test  output  on  the  record  should  be  readily  com- 
parable with  the  standaru  input  to  the  scoring  or  recording  mechanism. 

5.  Validity  of  Recording?  If  on  the  operational  device  the  trainee  would 
be  "on- target,"  does  the  iscor  on  the  synthetic  device  indicate  "on- target"  or 
vice  versa?  It  is  understenrt’  • Le  that  there  will  be  some  intrinsic  machine 
error  in  the  recording  by , ! a. ..  But  this  error  should  be  random  with  respect 
to  the  outputs  of  trainees  ~.«hcr  than  systematic  ana  thus  biasing.  (Random 
error  in  the  recording  sys+em  is  a matter  of  unreliability  rather  than  of 
validity. ) 

It  is  also  highly  important  that  the  recording  system  be  valid  for  all 
portions  of  the  scoring  range  or  segments  of  a behavior  path. 
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to  thTTSTif  the  lector.  “ J Sta*ri3i5^lSrS  and  controlled 

this  fact  in  mind.  Its  operation  should  be  rea  ly  ^ time-sharing 

on  . art  of  toe  r«t  of  the  tgiE-Sg^  2^2' In  the  chapter 
5K  other  taskFTs  involved.  Thio  matter  is  also  relerroa  uo 

dealing  with  the  instructor's  station. 
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CHAPTER  VI.  PROFICIENCY  MEASUREMENT 


file  primary  purpose  of  proficiency  evaluation  always  is  come  kind  of 
administrative  decision.  Thus  "Which  of  these  trainees  are  fit  for  the  job 
and  which  are  not?"  or  "Which  men  are  deserving  of  a promotion?"  or  "Which 
men  should  be  returned  to  training?" 

Job  proficiency  is  determined  with  perfect  reliability  and  validity  only 
by  evaluating  all  that  a man  does  during  his  lifetime  £2  or  at  the  job  itself. 
Any  other  "test"  of  proficiency  is  an  attempt  by  economical  means  to  select 
a sample  from  the  universe  of  job  problems  ho  is  going  to  face,  and  make  this 
sample  as  representative  as  tho  purpose  of  proficiency  evaluation  and  the 
budget  of  time  and  money  permit.  Other  things  being  equal,  the  wider  the 
range  and  the  greater  the  number  of  job  relevant  programs  (or  test  items) 
presented  to  the  tester,  the  better  our  prediction,  but  with  somewhat  dimin- 
ished returns.  Evaluation  is  not  an  end  in  itself:  it  is  meaningful  when  it 
predicts  future  job  performance. 

Absolute  vs. Relative  Criterion 


There  are  two  main  approaches  to  the  evaluation  of  proficiency.  An 
example  dealing  with  hardware  rather  than  personnel  may  highli^it  the 
difference. 

A customer  goes  into  a store  to  purchase  a calibre  .30  rifle.  He  states 
that  it  must  fire  with  a two  mil  gun  error  or  less.  The  shopkeeper  needs  do 
no  more  than  clamp  the  guns  in  his  stock  into  the  right  kind  of  vise,  sight 
the  gun  and  fire  a number  of  shots.  He  can  then  put  those  which  conform  to 
the  criterion  on  one  pile,  and  those  which  do  not  in  another  pile.  Presum- 
ably the  customer  will  now  accept  any  rifle  in  the  first  pile,  unless  he 
brings  additional  criteria  to  bear. 

This  is  a go/no-go  or  pass-fail  criterion  against  which  to  determine 
proficiency  or  acceptance  of  a product.  It  is  clean  cut,  but  depends  on  the 
customer  knowing  what  he  wants  or  needs.  Operations  analysis  or  its  equiv- 
alent may  help  to  define  human  ability  requirements  in  about  the  same  way. 

But  consider  another  situation.  The  customer  says,  "I  want  the  five 
most  accurate  calibre  .30  rifles  you  have  in  stock".  After  testing  the 
inventory  of  rifles,  none  of  us  knows  if  any  of  the  rifles  is  good  enough, 
or  if  on  the  other  hand  the  customer  is  getting  rifles  far  more  accurate 
than  he  needs  or  should  have  to  pay  for. 

Or  the  customer  may  ask  for  a rifle  with  a two  mil  error  or  less,  and 
no  rifle  may  meet  these  specifications.  Under  these  circumstances  the  cus- 
tomer may  legitimately  ask  for  the  five  rifles  which  most  nearly  meet  his 
need.  In  both  of  these  latter  cases  the  customer  was  accepting  a criterion 
based  on  supply  rather  than  demand;  and  a comparative  (group  norm)  rather 
than  absolute  criterione 
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In  roost  practical  cases  in  hardware  and  in  human  ability  requirements, 
there  tends  to  be  some  compromise  betwcon  the  pass-fail  kind  of  criterion 
and  the  group-norm  kind  of  criterion. 

The  Criterion  For  The  Job 


Proficiency  measurement  is  meaningless  unless  some  kind  of  criterion  is 
specified,  and  some  method  of  acceptance  and  rejection  made  explicit  whether 
it  be  on  the  basis  of  norms,  proportion  of  available  groups,  or  absolute 
quality  standards.  Obviously,  the  higher  and  more  rigid  the  quality  stand- 
ards, the  less  guarantee  of  sufficient  quantity,  and  vice  versa.  But  whether 
persons  be  selected  on  the  basis  of  absolute  criteria  or  on  the  basis  of 
group  norms,  proficiency  measurement  should  be  established  from  the  variables 
found  in  the  job  requirements. 

It  is  sometimes  necessary  to  use  the  same  method  of  measurement  for  de- 
termining not  only  pass-fail  groups,  but  for  dividing  a test  group  into  a 
larger  number  of  sub-groups.  Thus  we  might  want  to  separate  a total  class 
of  trainees  into  (a)  those  proficient  enough  for  operations,  (b)  those  not 
proficient  enough  for  operations  but  worthy  of  more  training,  and  (c)  those 
who  are  to  be  washed  out  of  training.  Additional  sub-groups  could  be  defined, 
such  as  those  who  are  to  be  transferred  to  a related  but  less  difficult  job 
or  training  task,  and  so  forth.  Ideally,  there  would  be  objective  criteria 
for  acceptance  into  each  of  these  groups,  and  these  criteria  would  be  set 
up  in  advance  and  independent  of  relative  performance  on  the  evaluation  test. 
This  ideal  is  usually  unrealistic  because  such  independent  criteria  are  rare- 
ly available.  Instead,  the  practice  is  to  measure  or  rate  performance  of 
the  group  members  on  the  evaluation  test,  then  put  the  top  100  men  into 
operations  (if  you  need  100  men  for  operations)  take  the  next  lower  25  men 
and  put  them  back  into  training  (if  you  have  room  for  the  25  men  in  training) 
and  wash  out  the  bottom  25  men  on  "general  principles"  or  because  you  don't 
know  what  else  to  do  with  theme  This  procedure  has  the  merit  of  simplicity 
in  that  it  avoids  the  difficult  problem  of  determining  exactly  what  perform- 
ance defines  success  in  operations,  or  defines  the  need  for  retraining,  or 
defines  non-worthwhileness  for  additional  training. 

In  any  event,  the  engineer  who  is  faoed  with  the  problem  of  designing 
a device  to  test  or  evaluate  proficiency  should  attempt  to  get  the  user  of 
the  device  to  specify  the  following: 

1,  Will  an  absolute  or  relative  criterion  be  used  in  determining 
what  will  happen  to  those  tested? 

2.  If  sn  absolute  criterion  is  to  be  used,  what  are  the  precise  per- 
formance requirements  (stated  in  behavioral  operations)  which  distinguish 
pass-fail  boundaries?  This  information  is  essential  to  efficient  designing 
of  scoring  and  recording  equipment,  and  to  the  selection  of  test  "programs" 
of  problems. 
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3.  If  pass-foil  criteria  are  not  available  or  planned;  what  basis 
will  be  used  for  dividing  those  tested  into  various  proficiency  groups 
or  levels? 

Many  other  considerations  go  into  the  design  of  proficiency  measuring 
devices . For  example,  are  some  job  outputs  or  consequences  more  important 
than  others?  If  so  what  are  they,  and  what  shall  be  their  relative  weights 
in  the  total  score  or  decision  to  be  made  about  the  trainee?  To  what  extent 
should  tho  variables  and  test  items  have  reference  to  selection  tests  given 
to  the  trainees  when  they  entered  training?  These  and  other  research  ques- 
tions are  important  to  resolve  before  undertaking  the  development  of  pro- 
ficiency measuring  devices.  Although  they  are  outside  the  scope  of  the 
design  engineer,  the  answer  may  have  considerable  bearing  on  the  kind  of 
equipment  which  is  built,  especially  with  respect  to  scoring  provisions, 
organization  and  programming  of  test  content,  and  the  provision  of  alterna- 
tive forms  of  the  test. 

Notice  that  the  approach  described  thus  far  emphasizes  the 
critical  aspects  of  job  outputs  determined  by  analysis  of  what 
the  man-machine  system  has  to  do.  The  logical  validity  of  this 
procedure  seems  clear.  There  is  another  approach,  however,  in 
which  an  attempt  is  made  to  reproduce  tho  physical  conditions 
of  job  operations  in  a synthetic  device  and  from  this  synthetic 
model  of  operations  derive  criteria  of  job  success.  Physical 
fidelity,  without  consideration  of  how  much  or  what  kind  may  be 
necessary,  is  used  in  the  attempt  to  guarantee  psychological 
fidelity.  The  designers  hope  that  through  a thorough  copy  of 
tho  job  itself  (except  that  inputs  and  machine  outputs  are  syn- 
thetic) the  pattern  of  job  success  will  reveal  itself. 

This  shotgun  approach  has  the  following  difficulties: 

1.  Physical  simulation  is  always  a matter  of  degree  in 
kind  and  scope  of  what  is  reproduced. 

2,  The  selection  of  "pro grams"  may  weight  variables  and 
behaviors  beyond  their  importance  in  actual  operations. 

3*  Programs  may  not  include  testing  the  full  range  of  the 
trainee's  ability,  especially  under  various  kinds  of 
load  which  may  arise  in  operations. 

li.  Thorough  physical  simulation  of  operational  conditions 
Is  extremely  costly,  especially  without  a guide  as  to 
what  comers  can  and  cannot  be  cut. 
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5.  The  inclusion  of  scored  items  which  are  non-diagnostic  of 
success  or  failure  in  operations  reduces  the  reliability 
of  the  overall  proficiency  evaluation  in  predicting  cri- 
terion performance  in  operations. 

Deciding  What  IP  Include  In  The  Evaluator 

The  same  general  considerations  which  go  into  the  decisions  of  what  to 
include  in  a trainer  also  go  into  the  decisions  of  what  to  include  in  an 
evaluator.  These  considerations  are  summarized  below. 

Let  us  presume  that  a complete  behavioral  task  analysis  has  been  made 
of  the  operational  job  (or  croup  of  related  jobs  if  a team  ia  to  be  evalu- 
ated). The  following  steps  should  be  taken  in  designing  the  evaluatort 

1.  Selection  of  tasks i The  first  step  is  to  decide  how  many  and  which 
ones  of  the  operational  tasks  shall  be  Included  in  the  devioe. 

Some  groups  of  tasks  may  be  tested  in  separate  devices,  althou^i  the 
selection  of  these  groups  should  be  based  on  considerations  discussed  above 
under  part-task  trainers.  chapter  III#) 

Thus,  if  tasks  are  time-shared  it  is  usually  risky  from  the  standpoint 
of  validity  to  test  them  outside  the  time-shared  context.  It  may  be  as  im- 
portant to  note  what  behaviors  or  task  performance  will  not  be  tested  by  a 
given  device  as  what  will  be  tested.  In  this  way  other  provisions  may  be 
made  for  evaluation  of  performance  on  the  other  tasks  or  job  segments. 

2.  Selection  of  sample  pro  grams i It  is  essential  that  the  programs  or 
problems  actually  presented  to  the  testae  be  at  least  representative  an5, 
if  possible,  exhaustive  of  the  kinds  of  critical  situation  with  which  he 
will  have  to  cope. 

Any  evaluation  must  be  based  on  some  sample  of  the  total  job  situations 
which  the  operator  will  encounter.  Thus,  to  give  a final  evaluation  of  a 
gunner  on  the  basis  of  one  shot  at  a single  kind  of  target  would  not  allow  us 
to  predict  how  well  he  would  do  on  different  kinds  of  targets  (to  say  nothing 
of  how  reliable  he  would  be  in  hitting  the  same  target  on  another  try) . 

Program  analysis  needs  therefore  to  have  been  made  of  the 
operational  jobj  in  fact  this  unalysis  should  have  been  part 
of  the  task  analysis. 

a.  Programs  should  be  varied  in  difficulty  as  well  as  in  kind  and  in 
combination. 

Other  things  (such  as  degree  of  learning)  being  equal,  the  greater  the 
number  of  things  the  operator  must  perceive  and  do  at  about  the  same  time, 
the  more  difficult  the  taskj  the  more  infomation  he  has  to  carry  in  his  head 
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at  about  the  same  time  in  immediate  Job  performance,  the  more  difficult  the 
task;  the  less  that  standard  routines  which  he  has  3s  arned  enable  him  to 
cope  with  a job  problem,  the  more  difficult  the  task. 

b.  Some  programs  should  be  included  as  test  items  which  go  considerably 
abovo  expected  operational  demands  in  order  to  explore  th-.  upper  limits  of 
the  operator's  capabilities,  especially  where  operational  pass-fail  criteria 
are  not  established, 

c.  Eliminate  non-diagnostic  material  from  test  "programs". 

Because  testing  time  is  usually  very  costly,  the  planning  of  programs  to 
put  into  the  evaluating  device  is  important  not  only  from  the  standpoint  of 
validity  of  testing,  but  also  in  economy  of  testing.  Customarily  an  18  hour 
operational  flight  has  many  relatively  "dead  spaces"  of  little  or  no  job  activ- 
ities. It  would  probably  be  uneconomical  to  simulate  3uch  dead  spaces  in  18 
hour  flights;  in  this  respect  the  copying  of  actual  job  situations  would  not 
be  efficient  testing  procedure.  We  should  remember  that  we  want  to  confine 
testing  to  those  requirements  of  the  job  which  do  differentiate  superior  from 
inferior  performers.  If  sustained  vigilance  or  resistance  to  fatigue  effects 
were  included  as  critical  job  requirements,  however,  testing  might  require 
18  hour  sessions  with  "dead  spaces." 

d.  Provide  test  material  that  is  highly  reproducible  and  standardized  from 
one  test  to  the  next. 

Unlike  the  demands  of  training,  proficiency  evaluators  must  have  hi^ily 
reproducible  programs  so  that  each  person  tested  is  exposed  to  the  same  stand- 
ard conditions  or  inputs.  Such  standardization  may  require  that  a good  deal 
of  the  programming  of  inputs  to  the  testee  be  automatic  rather  than  left  to 
the  option  or  manipulation  of  the  instructor. 

e.  Provide  alternative  standardized  programs  for  testing. 

Standardization  of  testing  conditions  introduces  another  problem  which 
must  be  met.  After  trainees  have  taken  tests,  word  gets  around  that  "an 
engine  fire  in  number  three  occurs  right  after  take-off"  so  that  persons 
tested  later  have  special  advantages.  Evaluation  tests  must  therefore  have 
alternative  forms  of  equivalent  difficulty.  The  equivalence  of  tests  can 
finally  be  determined  only  by  empirical  methods  of  actually  testing  matched 
groups  of  trainees  on  the  variations  of  the  test  and  making  statistical 
comparisons , 

If  the  test  has  comprehensive  coverage  of  the  job  requirements,  the 
"word  that  gets  around"  can  merely  be  that  the  test  covers  all  of  training,. 
Trainees  can  get  advantage  only  through  knowing  in  advance  what  sequence  of 
items  will  appear.  If  this  is  true,  the  only  problem  is  to  rearrange  the 
sequence  of  test  items  in  alternative  forms  of  the  examination.  Care  must 
be  taken  that  one  sequence  of  items  does  not  present  greater  difficulty  than 
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alternative  soquonco3. 

But  vhero  content  of  programs  (such  a3  in  navigation)  can  be  changed, 
such  changes  should  bo  made. 

f.  Pre-test  all  materials  on  a ’’pilot"  group  of  trainees  and  graduates. 

In  any  evont,  it  is  essential  that  pre-testing  be  made  of  the  equiva- 
lence of  forms  of  the  test  before  the  test  is  given  full-scale  use  in 
predicting  operational  job  proficiency.  Specialists  in  test  and  research 
design  can  aid  in  making  the  test  economical  and  efficient. 

What  variables  to  score;  The  proficiency  evaluator,  strictly  speaking, 
should  produce  information  about  the  trainee's  getting  jobs  done.  As  such, 
we  are  not  concerned  about  giving  the  trainee  useful  information  for  purposes 
of  learning;  we  are  concerned  primarily  with  deciding  whether  or  not  to  let 
him  undertake  the  real  job;  or  what  kind  or  level  of  responsibilities  to 
assign  to  him  in  operations. 

Consequently  our  interest  is  primarily  directed  not  to  the  responses 
the  trainee  makes  as  such,  but  to  whether  or  not  the  responses  he  feeds  into 
the  machine  produce  the  requisite  man-machine  output.  Do  the  bullets  or 
bonbs  hit  the  target  within  a given  acceptable  ratio  or  degree?  Does  he 
land  the  airplane  without  crashing?  Does  the  airplane  follow  an  efficient 
navigational  course?  Does  the  propeller  get  feathered  before  runaway  and 
without  damage  to  the  engine?  Is  the  enemy  intercepted  at  the  proper  time  and 
place?  1 

Notice  that  these  questions  are  about  primary  mission  objectives.  Se- 
condary objectives  may  be  derived  from  the  primary  objectives.  For  example, 
a pilot  may  perform  a series  of  maneuvers  very  successfully,  but  jam  his 
throttle  from  one  position  to  another  so  as  to  cause  unnecessary  wear  on 
the  engine,  or  be  wasteful  of  fuel,  ^ or  risk  killing  the  engine.  In  cases 
such  as  these,  the  response  itself  might  more  economically  be  noted  and  scor- 
ed directly,  rather  than  by  effect  (or  hypothetical  effect)  on  a machine. 

But  proficiency  data  obtained  in  this  way  are  bound  to  be  more  subjective  and 
may  sometimes  unduly  bias  the  final  evaluation.  Finally  the  score  must  yield 


^ Ideally,  these  determinations  would  have  been  made  of  the  trainee's  per- 
formance  with  the  "elimination"  of  machine  error,  (See  p.  96  below) 

^ Perhaps  the  concept  of  "amount  of  waste  produced  by  response”  should  be 
made  one  of  the  explicit  criteria  of  job  performance  and  recognized  as 
such  in  the  task  analysis.  Any  analysis  of  a complex  system's  efficiency 
is  related  to  a ratio  of  productive  output  to  waste,  and  the  same  approach 
might  be  taken  more  explicitly  to  the  evaluation  of  human  performance. 
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the  basis  for  making  a yes  or  no  response  to  the  question!  "Shall  the  man 
be  assigned  to  operations  or  not?"  — or  to  some  similar  type  question  calling 
for  a material  decision. 

The  quality  standards  and  quality  variables  of  any  man-machine  system 
generally  are  peculiar  to  that  syEtem  and  thus  specific  prescriptions  of 
these  variables  cannot  be  made  to  cover  a variety  of  jobs  and  systems.  The 
quality  variables  of  a lathe-produced  product  are  peculiar  to  the  needs 
which  that  product  must  serve;  thus  one  must  turn  to  the  needs  rather  than 
the  product  itself  for  standards  of  acceptability. 

Recommendation  in  determining  the  variables  to  score;  The  designer  of 
the  evaluator  should  attempt  to  determine  those  fewest  variables  of  machine 
output  which  are  common  to  all  or  the  greatest  number  of  programs  and  pro- 
blems to  which  the  man-machine  will  be  subjected  and  which  determine  adequacy 
or  inadequacy  of  overall  performance.  From  these  few  variables  refinements 
may  be  determined.  Ihe  designer  should  ask  the  question,  "What  does  infon- 
mation  about  this  variable  tell  us  with  respect  to  our  decision  about  accept- 
ing or  rejecting  the  testee  which  is  not  already  summarized  in  more  general 
variables?"  Such  questions  may  help  in  not  letting  the  scoring  problem  get 
out  of  hand.  Scoring  must  never  become  an  end  in  itself. 

Proficiency  evaluation  and  selection  testing;  Proficiency  evaluation 
may  sometimes  have  to  aid  in  justifying  or  modifying  selection  testing  pro- 
grams. If  this  is  true,  new  families  of  problems  and  considerations  may 
enter  the  picture.  It  is  important  to  keep  selection  test  validation  and 
research  problems  separate  from  those  connected  with  decisions  about  the 
fate  of  those  tested,  Ihe  use  of  proficiency  tests  as  tools  for  research  is 
outside  the  scope  of  this  discussion,  and  likely  to  be  outside  the  immediate 
responsibility  of  the  equipment  designer  except  insofar  as  design  require- 
ments are  already  imposed  on  him  through  external  channels. 

Scoring  And  Reference  Baselines 

Ihe  interpretation  of  scores  may  be  greatly  helped  by  automatic  refer- 
ence indications  telling  the  evaluator  when  the  performance  of  the  man-guided 
machine  is  within  tolerance  limits  and  when  it  is  not. 

Such  references  not  only  simplify  the  chore  of  interpreting  scores,  but 
increase  the  reliability  of  the  evaluating  process.  If  practicable,  the 
tolerance  reference  should  be  subject  to  change  by  the  evaluating  agency  in 
the  event  that  there  is  need  to  tighten  or  loosen  the  tolerance.  If  it  is 
not  practicable  to  make  a variable  tolerance  indicator,  then  it  should  be 
possible  readily  to  remove  the  fixed  tolerance  indication  from  the  record 
or  other  display.  Thus  the  interpreter  of  the  record  (or  other  scoring  dis- 
play) will  not  be  confused  by  the  presence  of  reference  marking  when  it  is 
inappropriate. 
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Excluding  Machine  Error  From  Scoring  In  Evaluation 

If  the  purpose  of  an  evaluation  procedure  or  device  is  to  measure  the 
effectiveness  of  a roan-machine  team  as  such,  the  operational  error  (or 
unreliability)  of  the  operational  machine  would  need  to  be  simulated. 

But  in  evaluating  the  proficiency  of  the  operator  or  trainee  as  such, 
the  evaluation  is  merely  made  more  unreliable  by  confounding  the  machine 
variable  error  with  human  error. 

That  is  to  say,  the  aim  of  proficiency  evaluation  is  to  get  as  reliable 
an  index  of  the  performance  of  the  man  as  is  practicable  on  a small  sample 
of  trials.  Assume  that  when  mam  X tries  a gunnery  pass,  the  machine's  error 
is  large,  but  when  Y tries  a pass  the  machine  error  just  happens  to  be  small. 
If  the  score  includes  machine  variable  error,  our  evaluation  of  X with  re- 
spect to  Y will  oe  biased. 

Even  when  we  are  concerned  with  an  absolute  job  criterion,  the  absolute 
criterion  of  human  performance  should,  if  possible,  be  determined  in  such  a 
way  that  machine  variable  error,  as  it  affects  a job  output,  has  been  elim- 
inated, (By  definition,  machine  variable  error  is  error  about  which  the 
operator  can  do  nothing!)  By  adding  it  to  a test  of  the  man,  any  individual 
measurement  about  the  man's  performance  has  its  reliability  diluted. 

If,  however,  the  unreliability  of  the  machine  is  a factor  with  which  it 
is  part  of  the  man's  job  to  cope  (such  as  in  maintenance  operations)  and  he 
can  make  or  learn  to  make  responses  which  do  cope  with  it,  then  that  type  of 
unreliability  should  be  represented  as  part  of  the  "programs"  which  test 
hi3  job  proficiency. 

Problems  of  measurement  can  be  unravelled  only  if  it  is  clearly  recog- 
nized and  decided  exactly  what  the  measurement  is  to  be  about  (the  man,  the 
machine,  the  man-machine  team,  ana  so  forth)  and  then  to  remain  consistent 
with  that  decision. 

Measuring  And  Evaluating  Human  Reliability 

It  is  often  important  to  know  not  only  a person's  average  performance 
on  the  job,  but  how  reliable  — or  how  variable  — is  his  job  performance 
from  one  assignment  or  mission  to  the  next. 

In  order  to  know  the  reliability  of  a man's  skill  it  is  necessary  to 
get  repeated  measurements  of  his  performance  on  equivalent  samples  of  task 
programs  or  problems.  If  the  same  problem  is  given  to  him  more  than  once, 
there  is  the  likelihood  that  some  specific  learning  effects  will  have  occurred. 
These  effects  would  contaminate  our  estimate  of  his  reliability  on  the  job 
where  programs  are  not  likely  to  be  exactly  repeated,  or  if  they  are,  the 
repetitions  are  interspersed  with  different  programs. 
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Py  getting  scores  on  a number  of  equivalent  items  (or  programs)  not 
only  can  the  mean  of  scores  bo  obtainod,  but  a measure  of  dispersion  such 
as  the  standard~3eviation  can  be  obtainod  for  the  individual,  (it  is,  of 
course,  common  practice  to  get  both  average  and  dispersion  scores  for 
groups  of  individuals  in  the  computation  of  norms.) 

Fbr  example,  we  may  find  that  the  average  circular  error  of  two  bom- 
bardiers is  100  feet.  But  the  first  man's  drops  cluster  within  200  feet  of 
the  target,  and  although  more  of  the  socond  man's  drops  are  bull's-eye  hits, 
he  also  occasionally  misses  by  1000  feet.  The  second  man  would  be  a better 
bet  to  bomb  the  enemy  troops  when  they  were  far  from  friendly  troops,  but  a 
poorer  bet  if  the  eneny  had  to  be  bombed  when  only  $00  feet  from  friendly 
troops. 

If  the  reliability  of  the  individual's  skill  is  to  be  determined,  vari- 
ations of  the  same  item  must  be  programed  in  the  evaluation  procedure. 
Provision  must  also  be  made  for  scoring  and  combining  scores  into  standard 
deviations  or  other  measures  of  dispersion.  Reliability  of  performance 
could  not  be  obtained  from  a counter  which  kept  only  an  accumulated  average 
of  performance  on  individual  test  items,  or  otherwise  pooled  scores. 


CHAPTER  VH.  THE  DESIGN  OF  THE  INSTRUCTORS  STATION1’2 


Introduction 


Designing  the  trainee's  station  is  devising  equipment  for  helping  Boaeone 
to  learn  to  operat^  other  equipment.  Bat  the  design  of  the  instructor's 
station  is  devising  the  work  space  and  equipment  it* elf  for  an  operator.  Thun 
it  involves  many  of  the  same  human  engineering  principles  which  apply  to  the 
design  of  an  airplane  cockpit  or  radar  operator* a panel.  There  are  come  special 
problems,  however,  to  which  specific  attention  may  be  directed. 

First,  lot  ua  enumerate  the  general  requirements  or  funotions  of  instruct- 
ion and  the  instructor* 

1.  Motivate  trainees  to  practice  and  learn. 

2.  Provide  guiding  information,  principles,  and  ideas  about 
doing  the  job. 

3.  Demonstrate  criterion  performance  if  necessary. 

1*.  Provide,  control,  and  monitor  the  stimulus  inputs  (pro- 
grams and  problems)  to  the  trainee. 

5*  Notice  the  trainee's  output  and  compare  it  with  criterion 
performance  ouch  as  in  scoring. 

6.  Provide  and  help  interpret  knowledge  of  results  to  the 
trainee. 

7.  Direct  trainee's  attention  to  critical  inputs  and  outputs 
at  various  times. 

8.  Control  general  conditions  of  practice  (such  as  how  much, 
when,  and  at  what  intervals;  adjusting  degree  of  task  dif- 
ficulty; integration  of  part  skills;  integration  of  train- 
ing into  operations). 


1.  This  chapter  is  a summary  of  the  report  by  Swain,  Alan  D.  Guide  for  the 
Design  and  Evaluation  of  the  Instructor's  Station  in  Training  Equipment, 
.bright  Air  Development  Center'  Technical  Report  In  many  places 

direct  quotations  have  been  made.  For  a complete  checklist  and  set  of 
recommendations,  the  reader  should  consult  this  reference. 


2.  Because  there  is  little  discursive  comment  or  "theory"  in  this  and  the 
following  chapters,  special  signals  denoting  Trainer  Becosmendations 
seem  unnecessary  and  are  therefore  omitted. 
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9.  Evaluating  trainee  perfonaance  with  respect  to  other 
trainees  or  an  operational  criterion. 

10.  Effecting  transition  from  instructor  dependence  to 
self-dependence. 

11.  Operating  the  mechanisms  of  instruction. 

At  least  indirectly,  all  of  these  functions  are  reflected  in  the  design 
requirements  of  the  instructor’s  station. 

The  block  diagram  on  the  next  page  schematizes  many  of  the  functions  and 
their  relationships  as  outlined  above. 


GENERAL  LAYOUT  OF  THE  INSTRUCTOR'S  STATION 

Every  layout  will  be  a compromise  between  setB  of  advantages  and  disadvant- 
ages. The  following  suggestions  are  intended  as  guidance. 

If  the  trainee  is  in  early  stages  of  learning  a complex  perceptual-motor 
and  decision-making  skill,  it  will  generally  be  desirable  for  the  instructor  to 
be  close  to  the  trainee  and  watch  him  directly.  Thus  he  can  guide,  offer  direct- 
ions when  the  trainee's  memory  falters,  instill  or  correct  scanning  habits  and 
eliminate  not  only  gross  errors  but  error  tendencies  through  direct  observation 
and  communication.  In  later  and  high  degrees  of  learning  a skill,  less  may  be 
lost  and  more  gained  if  the  instructor  is  more  remote,  and  receives  and  trans- 
mits information  through  instruments. 

Transition  training  provides  a special  case  where  large  amounts  of  con- 
fusion and  negative  transfer  may  be  experienced  by  the  skilled  operator  of  one 
device  who  is  being  transitioned  to  a similar  device.  For  transition  training, 
close  physical  proximity  of  instructor  and  trainee  is  desirable. 

If  the  expected  level  of  teaching  skill  and  experience  of  the  instructor 
is  low,  more  instrumental  aids  will  need  to  be  built  into  his  station  than  if 
he  is  highly  skilled  and  experienced. 

And  if  the  instructor  on  the  training  device  also  must  maintain  skill  on 
the  operational  device,  precautions  must  be  taken  so  that  his  operating  the 
trainer  does  not  degrade  his  skills  in  operations.  This  problem  can  easily  b8 
exaggerated,  however,  because  it  is  relatively  easy  to  learn  separate  habit 
systems  if  the  various  contexts  in  which  they  occur  arc  markedly  different. 


General  Equipment  Considerations 

The  trainee  should  not  be  able  to  see  the  instructor's  panel,  nor  see  (or 
hear)  the  instructor’s  movements  in  controlling  the  panel.  On  the  other  hand, 
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the  placement  of  the  instructor’ 8 station  should  not  interfere  with  the  dis- 
criminations, decisions  and  motor  responses  required  of  the  trainee  for  train- 
ing purposes.  Instruments  and  controls  most  used  by  the  instructor  should  be 
within  his  frontal  visual  field  and  within  easy  reach.  However,  where  the  lay- 
out is  complex  and  tho  instructor  has  time-shared  activities,  displays  and  con- 
trols should  be  grouped  on  different  panels  accordingly. 

The  instructor  should  be  able  to  aeo  as  much  of  the  trainee’s  work-space 
as  is  consistent  with  other  considerations.  At  least,  without  too  great  in- 
convenience, he  should  be  able  to  move  into  a position  from  which  he  may  ob- 
serve both  trainee  and  work-space. 

Symmetry  should  never  be  used  as  a principle  of  design  until  all  human  en- 
gineering considerations  have  been  taken  into  account 


Location  Of  Instructor  Relative  To  Trainee 


Tho  discussion  above  has  already  emphasized  when  it  is  desirable  to  have 
the  instructor  close  to  the  trainee.  It  is  probably  a good  general  principle 
(unless  one  has  poor  instructors)  to  get  as  direct  interaction  as  possible  be- 
tween instructor  and  trainee,  unless  the  instructor  provides  too  much  artificial 
information,  or  is  likely  to  misinterpret  the  trainee's  performance,  or  what  is 
important  in  the  trainee’s  behavior. 

Some  training  devices  may  provide  a number  of  alternatives.  The  instructor 
may  sit  so  as  to  observe  the  trainee  directly,  or  monitor  the  trainee's  outputs 
frean  a shielded  booth)  or  a combination  of  direct  and  indirect  contact  may  be 
designed.  This  will,  of  course,  depend  upon  the  nature  of  tasks  being  trained 
in  the  trainer.  For  example,  in  a visual  reconnaissance  trainer  the  instructor’s 
location  might  be  such  as  to  permit  him  to  observe  the  trainee  directly  and  to 
observe  the  trainee's  instrument  panel  and  the  target  (trainer)  presentation. 

Here  the  required  learning  is  primarily  perceptual. 


OPERABILITY  OF  THE  TRAINING  DEVICE 

The  training  device  should  be  designed  so  that  the  instructor  can  pay  maxi- 
mum attention  to  the  job  of  instructing  and  minimum  attention  and  effort  to 
monitoring  and  activating  the  device  as  such.  In  case  of  complex  trainers  it 
might  be  most  economical  to  have  trained  technicians  warm-up,  check,  and  main- 
tain the  device,  thus  leaving  the  instructor  free  to  devote  full  attention  to 
the  trainee. 

Checks  For  Operability)  Warm-Up  Procedures 

Determining  whether  the  trainer  is  operating  within  tolerances  should  be 
simple,  rapid  and  thorough.  Simplicity  of  calibration  and  adjustment  will  tend 
to  guarantee  that  the  device  will  be  correctly  calibrated  and  adjusted.  Fuses 
and  lights  should  permit  ready  access  and  easy  replacement. 
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Warm-up  procedures  should  be  simple.  There  should  be  automatic  provision 
against  damage  to  the  equipment  by  incorrect  wara-up  procedures.  Indications 
that  the  equipment  is  ready  for  use  should  be  clear  and  unambiguous,  although 
not  necessarily  centrally  located. 

Indications  of  on-off  status  should  be  clearly  visible  to  the  instructor. 
So  also  should  be  a master  indicator  of  the  over-all  operative  or  inoperative 
status  of  the  equipment. 


SCORING  AND  RECORDING  EQUIPMENT 


Problems  in  this  area  have  been  treated  in  Chapters  IV,  V,  and  VI  so  they 
will  not  be  restated.  It  may  be  emphasized,  however,  that  the  arrangement  of 
scoring  and  recording  equipment  should  take  into  account  the  time-sharing  ac- 
tivities of  the  instructor,  and  his  problems  of  interpreting  several  channels 
of  information  at  about  the  same  time.  Scaring  displays  which  should  be  read 
at  or  about  the  same  time  should  be  grouped  in  physical  proximity.  Scoring 
displays  presenting  information  which  needs  to  be  integrated  by  the  instructor 
should  either  be  grouped  in  physical  proximity,  or  distinguished  in  some  way 
(as  ty  color)  so  as  to  aid  in  a grouping  effect. 

The  presence  of  scoring  and  recording  equipment  which  is  not  used  acts  as 
visual  noise  to  the  instructor. 


HtSPIAT  OF  TRAINEE'S  PERFORMANCE  TO  INSTRUCTOR 


This  section  deals  loss  with  what  to  display  at  the  instructor's  station 
than  how  to  display  it.  Because  the  topic  of  display  design  is  treated  more 
fully  elsewhere,  only  a summary  treatment  is  offered  here. 

Perceptual  Simplicity 

The  v ay  in  which  channels  of  information  are  arranged  in  a work-space 
makes  for  relative  ease  or  difficulty  in  perception  and  use. 

The  first  principle  is  visual  accessibility.  All  displays  to  be  monitored 
by  the  instructor  should  be  visible  and  readable  from  his  position  with  the 


1.  Baker,  C.  A.  and  Grether,  W.  F.  Visual  presentation  of  information. 
M1DC  TR  5U-160,  Wright  Air  Development  Center,  Air  Research  and 
Denrelopaisnt  Command,  USAF,  Ifright-Fatterson  Air  Force  Base,  Ohio. 
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possible  exception  of  those  connected  with  turn-on  and  turn-off  procedure®. 

The  display  face  should  be  tilted  so  that  the  information  can  be  read  from 
the  normal  sitting  or  standing  position  of  the  instructor. 

The  grouping  of  displays  may  be  organised  around  one  or  aocre  of  the 
following  considerations  2 

Joint  monitoring  of  trainee's  and  instructor1 »s  etatiom  Does  the  location 
of  the  most  frequently  used  instruments  on  the  instructor  *0  panel  facilitate 
division  of  attention  between  displays  in  his  own  station  and  displays  in  the 
trainee's  station? 

Functional  grouping  t Instrument®  may  be  grouped  on  the  basis  of  spatial 
location  or  by  coding  them  ty  color,  size  or  shape.  The  basis  of  grouping  may 
be  time-sharing  or  reading,  or  they  may  be  grouped  according  to  tralneo  out- 
puts or  equipment  outputs  which  logically  are  associated  with  each  other. 

Sequential  grouping  i Displays  may  be  laid  out  to  follow  the  sequence  or 
pattern  of  operations  required  of  the  instructor.  Thus  if  the  instructor  fre- 
quently has  to  read  several  displays  in  a fixed  order,  they  could  be  l«id  out 
in  a straight  Line  and  according  to  the  sequence  read.  Because  of  the  relative- 
ly few  such  standard  sequences,  and  also  because  of  space  limitations,  sequen- 
tial grouping  may  not  often  be  feasible. 

Grouping  by  isolation:  A group  of  displays  can  be  perceptually  "isolated" 
by  enclosing  it  with  a white  line  on  the  panel,  or  by  using  fields  of  different 
colors  as  background.  More  than  a very  few  such  groupings  in  the  same  work- 
space, however,  may  create  more  confusion  and  visual  noise  than  aid. 

«n 

Operational  meaning i If  the  instructor  also  operates  the  operational  equip- 
ment, operational  displays  should  be  spatially  related  to  each  other  so  as  not  to 
interfere  with  his  normal  scanning  and  perceptual  habits.  Scale  may  be  reduced, 
but  relative  position  should  not  be  changed. 

Type  of  display:  The  same  frame  of  interpretive  reference  should  be  used 
as  much  as  possible  for  all  displays.  This  applies  to  the  units  used,  the  direct- 
ion in  which  the  pointer  moves,  moving  scales  vs.  moving  pointers,  and  so  on. 


Legibility  Of  Displays 

A number  of  items  should  appear  on  any  human  engineering  checklist  for  dis- 
plays with  respect  to  legibility.  These  itaas  include:  (a)  Identif lability  of 
the  display;  (2)  Size  of  display  relative  to  distance  viewed;  (3)  Illumination; 
(!■)  Numerals  and  letters;  (5)  Pointer  design;  and  (6)  Scale  design. 
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Other  Display  Considerations 

Sane  forma  of  display  movement  are  more  easily  read  than  others.  The 
more  easily  rend  are-  those  conforming  to  population*!  stereo  typos  (clockwise 
movement  of  pointer  to  indicate  increase,  upward  motion  of  pointer  indicating 
increase  in  magnitude,  and  so  forth).  Where  a conflict  of  such  stereotypes 
exists  in  a display  problem,  sane  alternative  solution  may  be  sought  if  time 
is  not  available  for  study  of  the  problem. 

Repeater  displays  at  the  instructor’s  station  are  especially  desirable 
when  the  inotrucloron  the  training  device  is  also  an  operator  of  the 
operational  equipment.  Repeater  displays  are  usually  less  expensive  to  install 
than  displays  which,  while  showing  the  necessary  information  about  the  inputs 
to  the  trainee,  are  specially  designed  to  present  the  data  more  simply  to  the 
instructor.  Repeater  displays  ore  also  desirable  when  the  instructor  is  re- 
quired to  determine  what  stimulus  (or  noise)  areas  the  trainee  in  responding  to, 
or  should  respond  to,  in  a display  which  is  otherwise  inaccessible  to  the  in- 
structor. For  example,  a radar  scope  at  the  instructor’s  station  which  is 
slaved  to  the  radar  observer  trainee's  scope  will  permit  the  instructor  to  ob- 
serve the  trainee’s  tracking  responses  during  fix  taking,  bomb  runs  and  wind 
determinations.  This  will  enable  the  instructor  to  forestall  the  development 
or  reinforcement  of  habits  involving  marked  perceptual  and  motor  response  er- 
rors. (In  this  case  the  "operator"  is  not  the  operator  of  the  equipment).  But 
before  it  is  decided  to  use  repeater  displays,  the  possibilities  of  complete 
revamping  of  the  displays  to  the  instructor  through  alternative  design  should 
be  explored.  Any  radical  simplification  would  likely  pay  off,  especially  if  the 
instructor  will  be  teaching  on  the  device  for  some  time. 

In  general,  if  the  instructor  is  also  an  operator  of  the  operational  equip- 
ment, rather  than  introduce  minor  changes  in  the  control-display  configuration 
and  relationships,  It  may  be  better  to  undertake  major  re-design  to  emphasize 
to  the  instructor  differences  in  training  and  operations  contexts  and  thus  re- 
duce habit  interference. 

Reduced  delay  between  trainee's  output  and  data  to  instructor: 

In  teaching  of  tasks  in  which  there  is  appreciable  time  lag  (more  than 
three  or  four  seconds)  between  the  trainee's  response  and  a normal  feedback,  it 
is  especially  desirable  for  the  instructor  to  get  immediate  data  about  the 
trainee's  response  ahead  of  toe  normal  feedback.  This  elimination  of  time  lag 
obviously  occurs  when  toe  instructor  is  able  directly  to  watch  the  trainee  at 
work.-  as  well  as  observe  toe  displays  and  the  instrumental  feedback.  The  in- 
structor can  then  provide  supplemental  feedback  or  guidance  more  synchronous 
with  toe  inadequate  response,  and  thus  facilitate  toe  elimination  of  toe  in- 
correct response. 
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THE  INSTRUCTOR’S  CONTROLS 


Consider at ions  about  controls  will  tend  to  parallel  many  of  those  under 
displays.  As  in  that  section,  treatment  here  will  be  summary  because  of  the 
material  on  the  deeign  of  control  systems  which  is  available  in  the  controls 
section  of  the  Joint  Services  Hisaan  Engineering  Guide  to  Equipment  Design,  (in 
preparation. )1  ~ 

Accessibility  and  location  should  be  based  on  frequency  of  useBj  avoiding 
the  likelihood  of  inadvertent  manipulations  j and  avoiding  the  need  for  the 
operator’s  hands  to  cross  while  manipulating  more  man  one  control  at  a time. 

As  in  displays,  controls  can  be  grouped  according  to  functxon  or  bo  se- 
quence of  use,  or  controls  can  be  isolated  (as  by  border  or  common  color,  or 
operational  meaning).  Unless  there  are  opposing  reasons,  population  stereo- 
types of  control  movement  (for  example,  turn  clockwise  to  increase  magnitude 
on  a moving-indicator  fixed-scale  display)  should  be  incorporated.  Reduction 
of  errors  is  likely  if  all  control-display  relationships  are  in  the  Bame 
direction,  unless  conflicts  with  previous  habit  patterns  of  the  instructor  will 
arise. 

Controls  should  be  placed  close  to  their  related  displays,  and  usually 
directly  under'  the  display^ 

Controls  not  relevant  to  the  instructor's  job  should  not  be  in  his  work- 
space. If  a number  of  control  functions  can  be  ganged  into  a single  control 
without  sacrificing  whatever  the  instructor  is  required  to  control,  such  gang- 
ing is  desirable. 

Coding  of  controls  can  be  done  by  one  or  a combination  of  tactual,  kines- 
thetic and  visual  variables.  If  the  instructor's  station  is  complex,  (as  for 
example  in  an  aircrew  training  simulator)  attention  to  coding  and  grouping  of 
controls  with  a view  to  reducing  instructor  load  and  error  probability  is  like- 
ly to  pay  off.  Instructor  error  in  control  manipulation  can  be  expensive  in 
training  time,  simulator  time,  and  in  cost  of  trainee  morale  and  confidence  in 
trainer  and  instructor;  it  may  also  result  in  faulty  training.  In  the  case  of 
proficiency  evaluation,  irreparable  damage  can  be  done  by  instructor  error  in 
a test  for  which  alternative  forms  of  the  test,  or  additional  testing  time,  are 
not  available. 

Division  of  control  labor  between  the  limbs  should  be  attempted.  If  the 
instructor  is  required  to  write  comments  or  evaluations,  provision  might  be  made 
for  the  left  hand  and  the  feet  to  share  heavily  in  control  activation. 
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The  many  considerations  dealing  with  control-display  ratios  and  control 
loadings  should  be  obtained  from  the  following  sources t 

Chapanis,  A.,  G&rnar,  W.  R.,  and  Morgan,  C.  T.  Applied 
Experimental  Psychology,  H.  I.:  Wiley  & Sons,  l5Ii?7 

Jenkins,  V.  L.  Design  Factors  in  Knobs  and  Levers  for 
naking  Settings  on  Scales  and  Scopes  (A  Sunmary  Report). 

WADC  TR  $3-2,  Wright  Air  Development  Center,  Air  Research 
and  L'jvolop'uent  Command,  USAF,  Wright-Patterson  Air  Force 
Base,  Ohio,  1953* 

Swain,  A.  D.  oj%  cit. 

Woodson,  W.  E.  (ed)  Roman  Engineering  Guide  for  Equipment 
Designers.  Human  Engineering  Section,  Italian  Factors 
Division,  U.  S.  Naval  Electronics  Laboratory,  San  Diego, 

Calif.  (University  of  Calif.  Press  195^) 

Repeater  controls  do,  however,  deserve  special  mention  here.  (Repeater  con- 
trols  are  linked  tandem  to  trainee  controls.)  Repeater  controls  for  the  purpose 
of  teaching  the  trainee  through  manual  guidance  are  not  likely  to  be  worthwhile, 
because  almost  no  learning  effects  occur  in  this  way.  It  is  extremely  doubtful 
if  repeater  controls  have  training  value  even  for  demonstrating  the  proper  use 
of  controls.  When  a trainee's  limbs  are  passively  moved  by  an  outside  force 
through  a configuration  which  is  supposed  to  bo  timed  to  a display,  learning  ef- 
fects are  generally  negligiblo. 

Repeater  controls  in  a training  aircraft  serve  a real  function  in  reducing 
hazard  to  trainee  and  instructor,  or  to  free  the  trainee  from  attention  to  con- 
trols while  observing  something  else  important  for  him  to  notice. 


TRANSMITTING  INFORMATION  TO  THE  TRAINEE 


In  this  section  "information  to  the  trainee"  will  include  not  only  speech 
from  instructor  to  trainee,  but  also  instrumental  inputs  to  the  trainee  controlled 
by  the  instructor.  Such  inputs  may  be  signals  about  the  learning  process  (for 
example,  a special  light  signalling  the  trainee  to  scan  more  than  he  is  now  doing) 
or  they  may  be  prog  .’am  inputs  such  as  emergencies,  target  courses,  and  so  forth. 


Speech  Communcatlon 

Direct  voice  communication  has  a number  of  advantages.  It  provides  rapid 
feedback  or  guidance  in  the  context  of  trainee  behaviors.  It  is  normally  less 
distracting  and  requires  less  effort  than  the  use  of  intercommunication  equip-, 
meat.  Direct  voice  is  of  disadvantage  in  communicating  over  a distance,  thus  it 
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fallows  the  instructor  somewhat  less  flexibility  of  movement  than  a portable 
microphone;  it  may  be  unsuccessful  in  competing  with  background  noise;  and  it 
does  not  provide  the  trainee  with  practice  in  the  use  of  interphone  equipment 
if  ouch  equipment  must  be  used  in  operations. 

If  intercom  is  used,  arrangements  should  free  the  instructor  and  trainee 
from  manual  control  if  practicable. 

If  more  than  one  instructor  operates  a training  device  at  the  same  time, 
it  is  desirable  that  they  intercommunicate  without  any  cues  being  perceived 
by  the  trainee. 

Speech  or  other  signals  having  to  do  with  the  learning  process  Bhould  be 
presented  so  as  to  minimize  disruption  in  ongoing  activities.  Sane  kind  of 
low-level  warning  that  such  a communication  is  imminent  may  be  provided  a mo- 
ment or  two  before  the  message  begins.  Thus  the  trainee  may  prepare  to  divide 
his  attention  without  blowing  up.  A two  to  three  second  foreperiod  is  usually 
optimal  for  simpler  tasks,  several  seconds  more  if  the  trainee  is  engaged  in  a 
difficult  (to  him)  task. 


Coded  Information  To  The  Trainee 


Cues  artificial  to  the  task  may  be  used  to  Bignal  the  trainee  to  perform 
certain  activities  he  has  been  neglecting,  or  provide  other  supplementary  in- 
formation. Some  examples  follow: 

1.  A special  tel-light  above  the  trainee's  panel  which  when 
activated  by  the  instructor's *bcan  switch11  tells  the  trainee 
to  "SCANl"  The  placement  of  this  light  should  be  somewhat 
outside  the  trainee's  normal  scan  pattern  so  as  not  to  pro- 
vide an  "action  cue"  artifical  to  operations. 

2.  Wheat-grain  lights  for  some  of  the  more  important  trainee 
station  displays.  Indirect  light  from  these  lamps  on  certain 
critical  trainee  displays  might  be  used  by  the  instructor  to 
remind  the  trainee  to  attend  to  them  at  certain  times.  The 
more  such  artificial  lights  beyond  four  or  five  (and  the  more 
they  are  used)  the  less  effective  they  become  for  training. 

3.  The  instructor  flashes  a correct  point  of  aim  in  a non-computer 
gunnery  device. 

ii.  The  instructor  usee  an  over-ride  switch  and  shows  the  correct 
"picture"  on  the  trainee's  radar  scope. 
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5*  The  instructor  flashes  a thin  beam  of  light  or  a light 
^circle'1  on  the  panoramic  target  presentation  in  a visual 
reconnaissance  trainer  to  point  out  target  stimulus  cues. 


(Note;  Care  must  be  exercised  lost  any  of  these  devices 
provide  crutches  to  the  trainee  which  will,  in  the  long 
run,  decrease  rather  than  increase  effective  learning  of 
operational  tasks.) 


Freeze  Switch 

Consideration  should  be  given  to  the  training  value  of  a freeze  Bwitch 
which  stops  and  holds  all  training  device  displays,  and  iS  feasible  from  an 
engineering  standpoint,  the  positions  of  all  controls  at  the  trainee's  station. 
During  such  a freeze  the  instructor  can  point  out  specific  display-control  re- 
lations which  the  trainee  should  or  should  not  be  making.  This  arrangement  has 
many  advantages  as  cited  under  optimal  conditions  for  Knowledge  of  Results 
(Chapter  IV). 

Discontinuous  Versus  Continuous  Control  Settings 

In  general,  a multi-position  selector  switch  is  better  than  a continuous 
adjustment  knob  for  inputs  to  the  trainee  station.  Thus  wind-direction  and 
velocity  usually  need  not  be  continuously  variable  throughout  their  range.  A 
discrete  number  of  settings  reduces  the  instructor's  load  and  likelihood  of 
error. 

Trouble  Consoles 


Training  in  complex  devices  often  teaches  the  trainee  how  to  detect  and 
cope  with  equipment  troubles,  emergencies  and  other  complex  program  inputs. 

To  the  extent  that  the  instructor  must  not  only  monitor  the  trainee's  perfor- 
mance but  control  complex  program  inputs  to  the  trainee  there  is  increasing 
danger  of  overloading  so  that  the  instructor  fulfills  neither  purpose  effect- 
ively. 

One  solution  is  to  provide  two  instructors,  one  of  whom  controls  the  pro- 
gram of  inputs,  and  the  other  monitors  trainee  performance. 

Another  solution  is  to  provide  gang  or  program  switches  and  single  con- 
trols which,  when  activated  release  a canned  sequence  of  program  inputs  to  the 
trainee. 

Or  it  may  be  made  possible  for  the  instructor  to  pre-set  a number  of 
troubles  at  his  panel,  and  then,  provided  with  holding  switches,  release  these 
troubles  at  times  best  suited  for  training  the  trainee  during  the  practice 
session. 
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(Tho  designer  3hould  again  bo  reminded  that  a largo  number  of  different  canned 
programs  should  bo  available  so  that  traineo  can  got  general  skills  at  coping 
with  a typo  of  problem  whatever  tho  combination  of  conditions  under  which  it 
may  occur. ) 

Coding  of  trouble  controls  and  displays  may  aid  the  instructor  to  dif- 
ferentiate troubles  which  tho  trainoe  should  have  responded  to  at  once 
(red  tel-light);  those  which  warn  the  trainee  of  a developing  troublo  (yellow 
tel- light);  and  those  which  indicate  an  inefficiency  of  the  man-machine  sys- 
tem due  to  improper  trainee  action,  but  which  is  not  a threat  of  emergency 
(blue  tel-light). 

Return  to  normal  operations:  After  a trainee  has  performed  a mal- 
practice the  equipment^  return  to  "normal"  operations  should  be  rapid  and  under 
the  control  of  tho  instructor.  Not  only  is  training  time  saved,  but  the  trainee 
has  opportunity  to  try  making  the  correct  response  before  he  forgets  what  he  did 
incorrectly. 

Either  the  equipment  should  return  to  "normal"  operations  automatically 
after  a certain  interval  of  simulating  a trouble,  or  the  instructor  should  have 
vivid  indication  (as  by  flashing  light)  that  the  trainer  is  3 till  simulating  the 
trouble. 


Sequence  Of  Trainee  Error  And  Instructor-Controlled  Inputs 

There  may  not  be  automatic  feedback  response  to  trainee  error.  For  example, 
the  trainee  may  forget  to  open  cowl  flaps.  It  may  then  be  appropriate  that  the 
cylinder  head  temperature  go  up.  If  the  instructor  must  manually  control  the 
indication  of  cylinder  head  temperature,  he  should  be  given  a special  signal  at 
his  station  that  the  trainee  has  neglected  to  open  cowl  flaps.  A master  light 
of  trainee  error  may  be  centrally  located  in  the  instructor’s  station.  His  at- 
tention then  is  referred  to  a special  "trainee  error  panel"  on  which  a specific 
tel-light  indicates  the  specific  error. 

Control  Of  Artificial  Feedback  Cues 


Operational  feedback  may  sometimes  be  supplemented  by  artificial  cues  in- 
forming the  trainee  when  he  is  or  is  not  successful  in  some  performance.  These 
cues  should  be  intermittent,  not  capable  of  being  anticipated  by  the  trainee, 
and  under  the  control  of  the  instructor. 

Control  Of  Continuously  Changing  Inputs  To  Trainee’s  Station 

"A  continuously  changing  input"  may  l ) one  which  literally  does  change 
from  moment  to  moment  such  as  an  enemy  target  aircraft,  or  it  may  also  be  one 
which  theoretically  may  be  changing  continuously,  such  as  wind  direction  and. 
velocity,  but  for  practical  purposes  changes  infrequently  and  more  or  less  dis- 

continuously. 
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Other  inputs  which  may  bo  on  the  borderline  between  continuous  and  dis- 
continuous are  enemy  jareaing  effects,  static,  and  other  types  of  visual  or 
auditory  stimuli  which  represent  noise  in  task  programs. 

If  well-trained  instructors  will  be  using  the  training  equipment,  it  is 
probably  desirable  to  have  manual  controls  for  these  inputs  to  the  traineo 
available  for  the  instructor,  even  if  automatic  or  canned  program  presentations 
are  also  available.  Flexibility  is  thus  possible  in  programing  for  a given 
trainee  at  a given  time.  The  instructor  can  keep  the  trainee  loaded  but  not 
overloaded;  he  can  repeat  certain  sequences  in  a program  or  otherwise  work 
intensively  with  certain  deficiencies  in  the  trainee  and  not  waste  time  with 
excessive  training  on  what  the  student  can  already  do  successfully. 

In  any  event,  the  instructor  should  at  will  be  able  to  cut  out  or  cut  in 
programed  noise  as  a trainee  input.  Operational  proficiency  should  be  estimated 
not  merely  on  how  well  the  trainee  performs  under  "representative"  operational 
conditions,  but  how  well  he  does  under  extremo  conditions  to  be  met  in  opera- 
tions short  of  complete  breakdown  of  (operational)  equipment  functioning.  (It 
is  also  generally  good  for  morale  in  training  that  the  trainee's  work  load  be 
near  his  capacity  during  any  stage  of  practice.  It  should  be  remembered,  how- 
ever, that  magnitude  of  load  during  learning  is-  in  part  self-imposed  by  the 
trainee:  It  is  determined  by  what  the  trainee  is  trying  to  do— the  range  and 
degree  of  error  he  is  setting  himself  to  avoid  or  reduce.) 

The  introduction  or  cessation  of  noise  signals  should  not  be  so  standardized 
that  they  become  action  feedback  cues  to  a trainee  during  practice. 

This  and  other  problems  of  artificial  restruction  of  input  programs  to  the 
trainee  in  training  may  be  reduced  by  providing  the  instructor  with  a detailed 
syllabus  which  itemizes  program  sequences.  Flexibility  permitted  in  the  sylla- 
bus should  not  permit  the  instructor  to  ignore,  during  training,  program  vari- 
ables or  situations  of  major  importance.  In  order  to  maintain  balance  in  train- 
ing, it  may  be  desirable  that  canned  or  taped  programs  be  presented  in  their 
entirety  to  the  trainee  in  occasional  practice  sessions.  Such  a session  might 
be  both  a training  session  and  a stage  of  proficiency  evaluation. 

If  the  simulated  target  is  operationally  maneuverable,  and  its  maneuvers 
are  in  part  a response  to  the  trainee-operator's  output,  a new  order  of  design 
problems  arises.  Automatic  target  course  generators  or  canned  target  programs 
become  prohibitively  complex  or  actually  unfeasible  from  an  engineering  stand- 
point, unless  the  response  of  the  trainee  should  be  standard  and  predictable. 

In  the  case  of  aerial  fixed  gunnery  tactics,  this  is  not  the  case;  the  action 
of  the  trainee-operator  and  enemy  are  partly  dependent  and  partly  independent 
of  each  other.  Since  flexibility  and  judgment  are  required  in  the  generation 
of  such  target  courses,  it  seems  desirable  to  assign  to  a second  instructor  the 
task  of  manually  controlling  target  movsnent.  Whether  or  not  this  control  should 
be  filtered  through  the  simulated  characteristics  of  the  enemy  aircraft  is  a pro- 
blem related  to  the  discussion  in  Chapter  III,  Stage  of  Learning  and  Degree  of 
Physical  Simulation,  pp.  k&*52c 
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PROVISION  FOR  INSTRUCTOR  DEMONSTRATION  OF  PERFORMANCE 


Thsre  are  some  provisions  for  demonstration  by  the  instructor  which  will 
be  more  fruitful  than  others.  Since  one  learns  what  one  does  (actually  or 
symbolically),  demonstration  may  constitute  symbolic  practice  of  a sort  depend- 
ing on  the  kind  of  communication  which  the  instructor  can  make  to  the  trainee, 
and  the  trainee's  capacity  later  to  transform  symbols  into  action.  Highly- 
skilled  pilots  can  learn  what  to  do  from  a briefing  session  in  which  maneuvers 
are  demonstrated  by  hand  movements  in  the  air.  But  the  pilot  has  already 
learned  the  many  component  responses  which  can  make  the  airplano  behave  in  the 
way  summarized  and  symbolized  by  the  hand  movement. 


Demonstration  Of  Motor  Aspects  Of  A Task 

The  instructor's  demonstration  to  the  trainee  of  the  motor  aspects  of  a 
task  produces  small  if  any  learning  benefits.  The  trainee  may  get  some  idea  of 
timing  of  response  with  respect  to  other  responses,  or  of  smoothness  of  response, 
but  these  data  may  be  useless  when  the  trainee  has  to  match  the  response  to 
task  cues  unless  he  already  has  the  response  patterns  in  his  repertory. 

Like  any  other  form  of  symbolic  ccranunication,  demonstration  may  be  of 
special  help  in  showing  the  sequence  of  responses  in  a procedural  task.  Danon- 
stration  will  tend  to  cut  down  the  trial-and-error  behavior  of  the  trainee. 

But  it  may  be  more  profitable  for  the  instructor  to  give  the  trainee  verbal 
guidance  while  the  trainee  himself  goes  through  the  response  sequence. 

In  conclusion,  provision  for  the  instructor  to  give  demonstration  of  motor 
performance  is  not  important  except  to  convince  the  trainee  that  criterion  per- 
formance required  in  "training  can  potentially  be  achieved  on  the  trainer. 

Demonstration  Of  Perceptual  Aspects  Of  A Task 

Demonstration  of  perceptual  aspects  of  a task  are  likely  to  be  of  relatively 
little  advantage  over  verbal  description.  There  are  some  exceptions.  The  demon- 
stration of  on- target  position  may  provide  a reference  for  the  trainee  which, 
although  subject  to  limitations  in  recall,  may  be  of  important  temporary  benefit. 
The  demonstration  of  what  a "good  picture"  looks  like  in  adjusting  a scope  may 
have  similar  benefit.  Provision  may  be  desirable  for  enabling  the  instructor  to 
demonstrate  a "good  picture"  without  major  disruption  either  of  practice  or  the 
instructor's  monitoring  of  practice.  If  the  instructor  is  seated  beside  the 
trainee,  he  can  manipulate  the  trainee's  controls  directly.  If  he  is  remote  from 
the  trainee,  he  may  be  about  equally  as  effective  by  slowing  or  stopping  any  pro- 
gram input  to  the  trainee  and  then  giving  verbal  instructions  which  get  the  trainee 
to  manipulate  the  controls  until  a good  picture,  or  an  on- target  display,  is  ob- 
tained. 
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GENERAL  FACILITIES  FOR  THE  INSTRUCTOR 


These  considerations  deal  with  the  conditions  which  promote  instructor 
comfort,  and  minimize  distraction  and  fatigue  effects.  They  are,  however, 
standard  problems  of  human  engineering  which  are  thoroughly  treated  elsewhere. 
They  are  summarized  as  follows: 


Seating  Arrangements' 

Standing  even  for  short  periods  of  time  is  fatiguing.  Seating  arrangements 
should  be  provided.  If  the  instructor  has  to  be  on  his  feet  and  move  around  a 
good  deal,  a high  stool  upon  which  he  can  rest  briefly  without  hoisting  himself 
up  and  down  may  be  more  desirable  than  a chair. 

Instructor's  Chair 


A person  who  spends  long  work  periods  in  a seated  position  deserves  a chair 
which  is  both  comfortable  and  functional.  Uncomfortable  contraptions  such  as 
jump  seats  quickly  produce  fatigue  and  may  in  themselves  be  almost  sufficient  for 
a person  to  get  to  dislike  his  job.  (See  Naval  Electronics  Laboratory  Human 
Engineering  Guide  for  Equipment  Designers^-  for  anthropometric  measurements  and 
specific  reccranendations  for  chair b.) 

Disturbing  Noise  In  The  Instructor's  Station 

If  the  instructor  can  monitor  auditory  and  tactual  noise  to  the  trainee's 
station  without  being  continuously  subject  to  that  noise  himself,  less  fatigue 
can  be  expected,  at  least  over  the  long  term.  Many  studies  have  shown  that  humans 
adapt  succeaafully  even  to  fairly  high  noise  levels,  but  there  is  some  evidence 
that  the  adaptation  is  at  a higher  energy  cost.  It  should  be  remembered  that  the 
instructor  spends  his  work  hours,  day  by  day,  at  the  same  job.  Conditions  which 
lessen  his  liability  to  annoyance  should  be  sought. 

Ventilation,  Temperature,  Humidity  At  The  Instructor's  Station 

These  factors  will  affect  the  instructor  as  they  do  any  other  operator.  The 
NEL  Guide  contains  information  on  ventilation  and  room  temperature  according  to 
amount  of  work  performed.  Chapanis,  Garner  and  Morgan  in  their  Applied  Experi- 
mental Psychology  offer  recommendations  on  temperature,  humidity  and  ventilation. 


1.  Woodson,  op.  clt. 
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Illumination 


The  illumination  at  the  instructor’s  station  should  take  into  account  not 
only  that  visual  field,  but  also  the  fiold  which  the  instructor  nay  have  to 
observe  in  the  trainee’s  station.  Harked  differences  in  general  illumination 
will  handicap  the  Instructor  by  dessanding  shifts  in  light  adaptation.  Direct 
sources  of  light  at  the  instructor’s  station  should  be  shielded  and  glare  sur- 
faces eliminated. 

For  general  requirements  of  absolute  illumination  for  various  types  of 
visual  work  see  Baker  and  Grether  (op.  cit. ). 

Grounding  of  Equipment 

The  training  equipment  should  be  properly  grounded  so  as  to  prevent  the 
instructor  or  trainee  from  being  shocked  by  static  electricity  or  power  voltages. 
Static  electricity  can  be  a source  of  considerable  annoyance  and  distraction, 
and  its  elimination  from  the  work-space  justifies  some  effort  and  expense  in 
design  and  construction. 


SUMMARY  OF  STEPS  IN  DESIGNING  THE  INSTRUCTOR'S  STATION 


The  following  steps  outline  a procedure  for  designing  the  instructor's 

station: 

Step  1.  Design  the  trainee's  station.  ThlB  design  includes  the  inputs  and 
outputs  of  the  trainee's  station.  Inputs  consist  of  variables  and 
channels,  and  programs  of  stimuli. 

Step  2.  Decide  which  inputs  (task  stimuli  and  knowledge  of  results)  to  the 

trainee  should  be  merely  monitored  by  the  instructor  and  which  shou.’.d 
come  under  his  active  control. 

Step  3.  Decide  which  outputs  (responses)  of  the  trainee  should  be  monitored 
by  the  instructor  and  which  are  not  necessary  for  him  to  monitor. 

Step  Make  a time-chart  of  the  instructor's  monitoring  and  controlling  tas1'* 
based  upon  the  time  chart  of  the  trainee's  Inputs,  outputs  and  respor'..* 
feedback  requirements. 

Step  5*  Decide  where  the  instructor  should  sit  in  relation  to  the  trainee's 
station. 
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Step  6.  Decide  which  uonitoring  and  controlling  aspects  of  the  instructor’s 
job  will  be  done  via  instructor’s  station  equipment  and  which  will 
bo  done  via  direct  observation  of  the  trainee  or  trainee’s  station 
equipment  and  by  voice  cocmmication  with  the  trainee. 

Step  7.  Decide  what  specific  displays  and  controls  are  needed  in  the  instruct- 
or's station. 

Step  8.  Lay  out  the  instructor's  displays  and  controls  according  to  optimum 
placement  for  his  required  functions  and  sequence  of  activities. 

Step  9*  Evaluate  and  modify  the  tentative  design. 


i 
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CHAPTER  VIII 

THE  TRAINER  iS  DEMONSTRATOR  OF  PRINCIPLES 


The  performance  of  a job  usually  requires  a combination  of  perceptual- 
motor  skills  and  problem-solving  or  ideational  skills.  The  learning  and 
transfer  of  perceptual-motor  skills  requires  practice;  the  learning  of  ideas 
and  ideational  skills  also  requires  practice.  Each  type  of  skill  improves 
more  rapidly  and  is  better  transferred  under  efficient  practice  conditions 
than  when  part  of  the  stimulus  inputs  and  response  outputs  is  irrelevant  to 
the  training  objective.  This  chapter  is  concerned  with  determining  what  is 
essential  and  what  is  non-essential  in  the  process  of  demonstration  which 
uses  such  equipment  asi  charts,  cutaways,  animated  panels  and  some  forms 
of  model  and  mock-up. 

Operational  equipment  and  complex  simulators  can  be  used  for  demonstra- 
tion and  part-task  trainers.  Our  discussion  in  this  chapter  will  be  directed 
to  devices  intended  to  be  used  for  no  more  than  demonstration. 


THE  FUNCTIONS  OF  DEMONSTRATION  IN  TRAINING 

Demonstrations  about  equipment  usually  shew  cause-effect  relationships 
between  activation  of  some  control  in  the  operational  equipment  and  some 
output  of  the  equipment.  Intervening  linkages  mey  be  "hewn  with  varying 
degrees  of  symbolism  and  simplification,  depending  on  the  relation  of  detail 
to  job  behaviors.  Some  principles  for  showing  input-output  relationships 
may  be  summarized  as  follows j 

A demonstration  should  simplify  concepts:  The  demonstrator  may  serve 
the  function  of  simplifying  the  idea  of  how  a complex  machine  or  piece  of 
electronic  gear  works  by  schematizing  its  linkages.  Its  advantage  over  a 
two-dimensional  drawing  is  that  the  trainee  does  not  have  to  imagine  the  dy- 
namic process  observed.  Thus,  the  push-pull  forces  of  a hydraulic  system 
may  bo  represented  without  showing  the  detailed  components  of  actuators  or 
the  electrical  circuitry  associated  with  servo  valves. 

Eliminate  details  not  relevant  to  principle  demonstrated:  No  effort 
should  be  spared  to  abstract  the  critical  action  or  worlcing  relationship 
and  ruthlessly  to  eliminate  superfluous  details  and  non-essentials.  For 
this  reason,  a good  diagram  or  blackboard  drawing  may  be  more  effective  than 
an  expensive  cutaway.  If  a number  of  variables  interact  to  affect  an  out- 
put (such  as  a bomb  drop)  the  influence  of  changing  each  of  these  variables 
should  be  clearly  demonstrated  by  a change  in  the  output.  Actual  magnitudes 
need  not  be  simulated  in  the  device  — » they  may  be  shewn  better  and  more 
simply  by  equations  on  a blackboard. 

Dramatise  output*:  It  is  generally  better  to  represent  a t}targetn  as  a 
bridge  or  marshalling1 yard  than  as  a black  spot.  This  representation  of  the 
literal  effect  of  an  output  is  important  not  only  for  arousing  immediate 
interest,  but  also,  by  being  concrete,  it  will  aid  the  trainee  in  transferring 
the  concept  to  real  operations.  That  is,  h®  will  remember  it  better  later  on. 
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This  suggestion  is  not  inconsistent  with  tho  previous  statements  about  simpli- 
fying tbs  presentation  of  a prooess,  because  it  deals  with  an  outcome  of . the 
process.  However,  if  inconsistencies  appear,  lean  towards  simplification 
rather  than  realism. 

Dramatise  differences  between  old  and  new  operations  in  transition  train- 
ing: Wrougji Emphasis  and  distortion  special  differentiation  can  bo  made  be** 
w»on  either  stismlus  components  or  response  components  in  a new  job  or  task 
which  differs  from  one  previously  practiced  by  tho  trainee. 

the  entire  demonstration  should  be  taken  in  at  a glance:  In  order  that 
input-output  relationships  can  be  dearly  understood^  the  soalo  of  the  demon- 
strator should  permit  a view  of  the  complete  aotion  to  be  Bhtsra,  On  tha 
other  hand,  the  soale  of  the  demonstration  should  not  be  so  s&all  that  changes 
in  outputs  seem  negligible  or  unimportant. 

Levels  of  understanding:  "Understanding*  how  something  works  is  not  an 
all- or- none  matter.  The  understanding  of  electronics  required  for  competence 
and  interest  in  his  job  by  a designing  engineer  may  be  (and  isi)  quite  differ- 
ent from  the  kind  of  understanding  required  by  the  mechanic  who  services  the 
equipment.  The  mechanic  may  only  operate  a few  switches  on  the  front  of  a 
checkout  sot  and  replace  black  boxes  when  certain  indicators  light  up.  Com- 
petent understanding  of  soil  chemistry  does  not  nsoessarlly  require  a ground- 
ing in  nuclear  physics. 

Tho  tore  "understanding"  should  therefore  have  subscripts  depending  on 
the  purpose  and  actions  to  be  related  to  each  other  by  each  understanding. 

At  some  level  of  operations  all  theoretical  understanding  may  become 
"functional"  understanding  — that  is,  the  knowledge  of  what  does  what  to 
what.  This  homely  phrase  suggests  that  a level  of  understanding  has  a cri- 
terion of  relevance.  Electrons  do  not  aot  (directly)  upon  molecules  — a 
discussion  of  energy  changes  has  different  levels  of  discourse  defined  trjr 
the  objects  which  interact  with  each  other. 

The  appropriate  level  of  understanding  at  which  the  training  curriculum 
and  demonstrations  should  be  pitched  may  best  be  determined  by  the  structur- 
al units  with  which  the  job  will  be  concerned  and  the  "principles"  which  are 
directly  applicable  to  predicting  the  interaction  of  those  units.  Functional 
understanding  will  be  defined  now  by  t«e  job  functions  of  the  trainee.  His 
understanding  should  bo  consistent  with  the  level  of  operations  and  decision 
he  will  have  to  make.  A man  working  with  dangerous  chemicals  should  know 
what  material  they  will  damage  and  the  proper  method  of  stopping  their  action, 
but  need  not  know  the  atomic  weight  or  valence  of  the  elements  which  the 
chemical  contains. 

The  presentation  should  herefone  involve  the  smallest  number  of  vari- 
ables and  their  interactions  which  are  consistent  with  the  trainee’s  interests 
at  this  stags  of  training  as  well  as  with  the  job  requirements  and  the  ability 
levels  required  for  that  job.  This  simplification  should  result  not  only  in 
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training  cccnosy;  but  increased  s«ns«  of  nnhiow^nt.  and  intorest  among  the 
trainees.  There  is  another  advantage  in  simplicity  of  training  demands 
consistent  with  job  requirements:  good  persons  for  the  job  will  not  have 
been  screened  out  during  training  because:  intellectual  requirements  were 
demanded  which  were  irrelevant  to  satisfactory  performance  on  that  job. 

The  Demonstrator  As  A Bmia  Fbr  Generalising  Information  And  Training 

The  principles  of  aerodynamics  apply  to  ary  aircraft  (except  for  acme 
discontinuities  as  at  sonic  speeds)  and  the  geometry  of  ballistics  apply  to 
any  projectiles.  The  functions  of  at  least  some  demonstrators  is  to  show 
vividly  and  simply  to  the  trainee  the  basic  operating  principles  of  a class 
of  machines  or  equipment. 

It  is  well  known  that  abstract  principles  can  usually  be  easier  learned 
and  better  remembered  through  concrete  examples,  if  the  examples  clearly  get 
over  the  point  intended. 

Demonstrational  devices  may  theroiore  show  to  a trainee  getting  either 
"generalized"  training  or  transition  training  the  operating  principles 
common  to  a class  of  machines.  Examples  are:  hydraulics,  aerodynamics, 
electronics,  servos,  ballistics,  mechanics,  turbines. 

The  demonstrator,  in  the  hands  of  a skilled  instructor,  may  aid  in 
teaching  for  transfer  of  training  at  the  conceptual  or  "idea"  level  of  job 
performance. 

The  Demonstrator  As  A Working  Framework  For  Relating  Separate  Areas  of 
Training 

A demonstrator  may  be  designed  partly  to  show  the  relationship 
of  one  portion  of  a training  course  to  another  and  thus  serve  to  bind  them 
together.  Training  aids  especially  designed  for  showing  the  various  inter- 
locking functions  of  a complex  electronic  device  may  serve  such  a purpose. 

In  effect,  it  will  help  to  unify  at  the  conceptual  and  idea  levels  the  sep- 
arate blocks  of  inforaation  the  trainee  has  or  is  learning  about  a system. 
Thus  if  an  operational  system  contains  eight  major  sub-systems,  it  may  be 
easy  for  the  trainee  to  forget  while  studying  sub-system  five  much  of  what 
he  has  learned  about  one  through  four.  Ey  relating  parts  of  knowledge  to 
a central  idea  (such  as^  the  system  acting  as  a whole)  the  parts  themselves 
may  be  better  remembered  and  better  used. 


SELECTING  THE  WORKING  COMPONENTS  OF  THE  DEMONSTRATION  TRAINER 

Decisions  as  to  what  to  include  and  exclude  in  the  design  and  construction 
of  a training  aid  or  demonstrator  should  be  based  principally  on  the  job  be- 
haviors of  a particular  job  and  the  ideas  or  mental  concepts  which  will  help 
get  that  job  more  quickly  learned,  or  better  done.  A demonstration  device 
should  show  simply  what  is  otherwise  complex  to  understand. 
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On  the  other  hand,  it  la  easy  to  exaggerate  hov  raich  can  be  learned  by 
demonstration.  Demonstration,  like  other  forms  of  guidance,  may  tend  to 
cut  down  the  range  of  trial-and -error  of  the  trainee,  but  it  doea  not  take 
the  place  of  actual  praotloe  either  in  problta-aolving  or  in  perceptual^ 
motor  performance.  Large  expenditures  in  glamorized  demonsTration  apparatus 
(such  as  complex  cutaways)  are  not  likely  to  be  justified  on  practical 
grounds.  As  elsewhere  in  training,  the  crucial  question  to  be  asked  iat 
"What  specifically  will  the  trainee  learn  or  do  better  on  his  job  because 
of  this  training  device?" 

Before  a demonstrator  is  constructed  the  question  should  be  asked* 

"In  what  specific  way  does  this  device  help  the  trainee  above  what  could  be 
shown  by  a crude  sketch  on  a blackboard?"  Because  demonstration  is  usually 
interpreted  through  symbols  (sentences,  ideas  and  so  forth)  by  the  trainee, 
the  representation  may  be  schematized  and  symbolic,  and  still  transfer  to 
the  trainee's  training  need. 

The  general  questions  from  which  training  aids  and  demonstrators  may 
be  logically  derived  for  training  purposes  include  the  following* 

1.  What  human  inputs  to  the  machine  are  required  by  the  job? 

2.  What  non-human  inpute  go  into  the  operational  equipment? 

3.  What  outputs  of  (and  within)  the  equipment  affect  job  outcomes? 

li.  What  are  the  main  transmission  linkages  in  the  operational 
equipment  between  the  inputs  and  outputs  which  are  affected 
by  the  operator's  job? 

A Semple  Problem 

Let  us  take  a sample  problem  and  sketch  our  way  through  it. 

We  have  examined  the  job  operations  and  tools  of  an  engine  mechanic 
and  recognize  a mechanical  principle  which  he  must  use  (wittingly  or  other- 
wise) in  performing  his  job.  This  principle  is  also  <m  important  on®  to 
the  design  of  the  equipment  on  which  he  works.  The  second  factor  is  less 
important  than  the  first,  but  it  is  worth  bearing  in  mind, 

Mary  job  demands  require  the  concentration  of  force  by  use  of  tools* 

We  see  that  the  use  of  screw-drivers,  jacks,  crowbars,  and  a number  of  other 
tools  have  in  common  the  principle  of  "leverage.'5  We  abstract  the  consnon 
features  of  activities  performed  by  these  tools.  They  include  a fulcrum, 
a rigid  member,  a point  of  applied  force  on  the  member  and  a point  of 
force  applied  to  an  object.  The  factor  critical  to  the  force  relationship  is 
the  relative  length  of  the  rigid  member  from  fulcrum  to  object  and  fulcrum 
to  hands 
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We  may  represent  the  principle  in  the  classical  test-book  diagram  thus: 


Schematic  Diagram  of  the  Principle  of  the  Lever 

Unless  the  trainee  is  quite  intelligent,  or  has  already  had  a course  in 
elementary  physics,  he  will  not  be  able  to  translate  this  abstract  diagram 
into  experience,  real  or  imagined.  It  won't  make  "sense0  to  him  because  it 
lacks  sensory  implications.  Furthermore,  ve  have  overloaded  the  diagram  with 
too  much  information  for  a simple  grasp  of  meaning. 

Let's  introduce  concrete  elements  into  the  diagram  and  instead  of  analyz- 
ing mechanical  advantage  in  one  diagram  take  one  variable  at  a time. 
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Avoid  unnocesaary  precision  in  conveying  an  Idea:  Since  the  mechanic 
will  hoi  be  required  in  the  course  of  his  job  to  solve  equations  beyond  the 
crude  level  of  determining  "more  than"  and  "less  than"  it  is  possible  that 
an  attempt  to  include  equations  on  these  diagrams  might  set  up  negative 
reactions.  If  the  matter  of  learning  to  solve  the  equation  is  considered 
of  sufficient  importance  (ai/d  the  job  requirements  probably  would  not  justi- 
fy it)  a moving  model  mi$it  b8  constructed  with  variable  fulcrum,  balance 
weights,  and  distance  scale,  ho  more  than  two  fulcrum  positions,  three 
weights  and  two  distance  indications  on  each  distance  scale  should  probably 
appear  in  order  to  keep  the  relationships  dramatically  simple,  and  in  order 
to  reduce  the  cost  of  manufacture  by  requiring  coarse  precision. 

Show  varied  examples  from  the  jobt  Ihe  next  step  in  the  series  of  dia- 
grams wb\A~d  consist  of  showing  the  ways  in  which  the  mechanic  applies  the 
principles  of  leverage  by  using  various  tools,  and  using  them  in  various  ways. 
Each  picture,  as  of  orovbar,  screw-driver,  wrench,  wing-nut,  and  so  on,  should 
show  the  eloments  of  applied  force,  fulcrum  position  and  exerted  force. 

Separate  diagrams  might  be  needed  to  schematize  an  applied  and  exerted  force 
on  the  same  side  of  the  fulcrum. 

Representing  Transmission  Systems 

If  the  trainee's  job  calls  for  action  or  decisions  based  on  trouble- 
shooting the  equipment,  the  transmission  linkages  or  loops  within  the  machine 
will  need  to  be  represented.  In  the  interests  of  economy,  simulation  of  the 
actual  mechanisms  or  portions  of  the  system  need  not  go  within  those  compon- 
ents whose  inputs  and  outputs  will  provide  the  information  needed  for  diagnosis. 
That  is,  if  the  mechanic  or  operator  in  the  course  of  his  job  will  not  conduct 
actual  operations  within  a given  black  box,  but  will  have  to  decide  whether 
or  not  this  box  is  satisfactory  for  operations,  the  input-output  functions 
of  this  box  in  the  trainer  may  be  simulated  by  mechanical  or  electrical  means 
other  than  those  used  in  the  operational  equipment. 

Whether  or  not  a three-dimensibnal  mock-up  (non- functional)  of  a complex 
equipment  is  as  good  a general  teaching  device  as  a functional  or  actually 
working  demonstrator  is  difficult  to  say.  It  is  even. more  difficult  to  say 
whether  a functional  demonstrator  is  worth  the  additional  cost  over  a mock-up 
in  terns  of  training  advantage.  It  Is  probably  the  skill  of  the  instructor 
which  is  the  critical  variable.  Certainly  the  prediction  of  the  probable 
usefulness  of  a demonstrator  cannot  be  made  without  reference  to  a syllabus 
on  its  use  in  training. 

A demonstrator  used  in  teaching  trouble-shooting  should  allow  malfunction 
sytsptoss  to  be  represented  as  they  may  appear  to  tho  operator  or  mechanic  or 
both;  The  source  of  the  symptom  should  be  traceable.  If  various  malfunctions 
nay  give  rise  to  similar  or  identical  symptoms,  the  linkages  between  the  vari- 
ous possible  causes  and  the  effect  should  be  clearly  manifest. 


-120- 


WADC  TR  5U-563 


Trcinsr  ss  demonstrator 


The  foregoing  discussion  suggests  that  tvo  seta  of  information  will 
bo  necessary  prerequisites  to  the  adequate  design  of  a demonstrator  'for 
training  in  trouble-shooting.  One  set  of  information  will  consist  of  what 
the  operator  or  mechanic  has  to  do  on  his  job  (action  or  decision)  involv- 
ing checking,  adjusting,  repairing,  or  replacing  the  equipment  or  parts  of 
it.  Another  sot  of  information  will  consist  of  malfunction  symptoms  char- 
acteristic of  the  operational  equipment.  The  representation  of  the  linkages 
between  symptoms  and  symptom  sources  according  to  Job  behaviors  then  will 
define  the  essential  components  of  the  demonstrator  for  trouble-shooting. 

Coding 

The  display  of  transmission  systems  and  sub-systems  should,  of  course, 
enhance  the  differences  between  those  parts  of  the  system  which  are  function- 
ally independent  of  each  other,  and  should  show,  by  suitable  coding,  the 
relations  of  parts  of  the  system  which  are  interdependent.  Coding  should  be 
consistent  throughout  the  demonstrator  and,  if  possible,  consistent  with  coding 
used  in  the  trainee’s  texts,  training  manuals,  and  so  forth.  Ideally  the  dis- 
play should  permit  instantaneous  perception  of  the  entire  set  of  operating 
relationships  being  presented  at  one  time.  These  relationships  should  require 
minimum  time  and  effort  to  switch  from  one  to  another.  TL,  relationships 
should  be  readily  obsarvable  by  all  the  trainees  ir.  whatever  group  is  working 
at  the  demonstration.  The  more  time  required  (beyond  a few  seconds)  to  get  a 
demonstration  under  way,  the  greater  the  boredom  and  loss  of  learning  motiva- 
tion. Besides  loss  of  interest,  there  is  also  loss  of  continuity  and  carry- 
over of  ideas  if  there  are  time  lags  between  one  portion  of  the  demonstration 
and  later  portions. 

Recommendations  for  Use 


A syllabus  should  bo  prepared  to  accompany  each  training  aid.  This 
syllabus  should  clearly  identify  the  job  behaviors  related  to  what  the  train- 
ing aid  is  intended  to  teach.  Suggestions  should  also  be  made  as  to  hew  the 
instructor  can  test  whether  the  trainee  has  acquired  and  will  transfer  the 
job  knowledge  obtained  from  the  demonstration  trainer. 
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CHAPTER  IX 

OUTLINE  OF  STEPS  IN  DESIGNING  A TRAINING  DEVICE 


Introduction 


The  following  pages  are  a summary  of  the  steps  to  take  in  the  systematic 
design  of  a training  device.  It  will  be  reasonable  for  the  designer  of  train- 
ing devices  to  have  the  collaboration  of  a specialist  in  the  psychology  of 
training  and  the  methods  of  task  analysis  summarized  below. 

How  to  go  about  performing  these  steps  has  been  described  in  summary 
fashion  in  the  preceding  chapters  and  in  greater  detail  in  the  references  cited 
at  the  end  of  the  chapter. 

1,  Prepare  a task  analysis  of  the  man-machine  job 

The  logical  first  step  in  the  design  of  a training  curriculum  or  of  a 
training  device  is  to  determine  what  the  operator  in  the  man-machine  job  has 
to  do.  This  step  follows  from  a determination  of  the  input  and  output  chan- 
nels of  the  man-machine  system  and  the  role  played  by  the  operator  (or  mechanic, 
if  it  is  a maintenance  job  being  studied)  in  these  linkages  or  system  functions. 
Determine  the  quality  variables,  tolerances  and  standards  required  of  the  man- 
machine  team  as  best  as  can  be  done. 

Distinguish  procedural,  discontinuous  tasks  (such  as  switch-turning)  from 
continuous  or  tracking  tasks  such  as  aiming  a weapon  or  driving  a vehicle  or  » 

craft. 

Itemize  tasks  awarding  to  sequence,  and  those  actions  performed  at  or 
about  the  same  time. 

Within  each  task  isolate  the  behavioral  steps  ...v.  . 1 . .tify  the  critical 
discriminations  (perceptions),  decisions,  memory  st or.-*;,  and  human  motor  out- 
puts required  of  the  operator  in  operating  and  monito.  Ing  the  equipment.  Of 
special  importance  for  each  task,  and  often  for  each  behavior,  is  the  Likely 
or  Characteristic  Error  or  Malpractice  which  can  be  expected  of  it.  Both  "the 
•’Likely  Error”  and  its  behavioral  background  may  be  derived  from  consider- 
able study  of  the  task,  the  work  environment,  sequence  of  behaviors  and  anti- 
cipations, and  from  wide  experience  of  the  analyst  in  behavior  patterns  and 
habit  systems.  Training  obviously  should  be  directed  against  error  tendencies, 

3ome  of  which  may  be  well  buried,  and  which  may  arise  only  in  the  development 
of  hi^ier  orders  of  skill  by  the  trainee  and  operator. 

Determine  the  concrete  "programs0  or  environmental  variables  and  condi- 
tions in  which  each  task  will  be  performed  in  operations.  t 

Prepare  a time-base  chart  of  mission  cycles  or  maneuver  cycles.  0 set, 
duration  and  overlapping  of  behaviors  should  be  represented  on  the  time- 
analysis  chart  of  behaviors.  This  chart  will  show  when  the  operator  is 
heavily  loaded,  the  number  and  kind  of  channels  of  information  to  which  he 
must  respond,  the  signals  which  he  must  anticipate,  and  his  demands  in  reaction 
time  9 
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Tho  preparation  and  interpretation  of  such  a chart  will  probably  require 
the  aid  of  a specialist  in  task  analysis  and  human  engineering.  ^ 

2.  Select  the  tasks  which  are  to  be  trained 


The  job  segments,  tasks  and  programs  for  which  the  trainee  is  to  be 
trained  on  a given  device  will  need  to  be  selected.  A number  of  these  de- 
cisions may  be  relatively  arbitrary.  This  selaction  should,  however,  be 
aided  by  a human  engineer  who  can  point  out  dangers  to  training  efficiency 
by  some  selection  of  tasks  from  the  total  task  context,  while  he  may  suggest 
training  device  economies  by  other  selections  of  tasks  and  programs  for  a 
given  trainer* 

Part- task  trainers  can  be  efficient  and  very  economical  in  a training 
program,  but  their  design  from  the  human  engineering  standpoint  requires  a 
hi$i  degree  of  sophistication  in  the  area  of  human  learning  and  transfer* 
Their  limitations  should  also  be  indicated  so  that  they  will  not  be  need 
inappropriately.  (See  Chapter  III.) 


3.  Decisions  on  degree  of  training  expected  on  tasks 

Having  decided  what  the  trainer  is  to  help  teach,  we  should  decide  how 
well  tho  trainee  should  be  able  to  perform  it  by  practice  on  the  trainer. 

Degree  of  training  is  important  as  it  relates  to  principles  and  considerations 
as  given  in  Chapter  HI,  "Stage  of  Learning  and  Degree  of  Physical  Simulation.” 
A trainer  may  be  very  useful  even  if  it  trains  some  tasks  to  less  than  opera- 
tional adequacy.  The  human  engineer  should  assist  the  design  engineer  and 
delegates  from  operational  and  training  agencies  in  making  these  decisions, 

li*  Assumptions  about  the  trainea  population 

Neft  we  will  want  to  know  what  kind  of  trainee  will  be  using  the  trainer. 
That  is,  what  relevant  skills,  knowledge  and  abilities  will  be  brought  into 
the  training  situation.  Estimates  of  trainse  load  will  help  put  trainer 
needs  into  perspective* 

5.  Assumptions  about  instructional  personnel 

It  is  helpful  to  know  what  skills  and  abilities  can  be  expected  of  the 
instructors  so  that  what  the  instructor  is  not  able  to  do,  the  equipment  can 
attempt  to  do  for  him.  For  example,  will  the  instructor  have  skills  in  the 
operational  equipment  for  which  training  is  intended?  Will  the  same  instruc- 
tor teach  the  trainee  in  the  trainer  and  on  the  operational  equipment?  Chapter 
VII,  "The  Instructor’s  Station,"  explains  why  this  knowledge  is  important  for 
design  of  the  trainer. 


1.  See  Appendix  I and  II  for  samples  of  task  analysis  pages  and  time -charts* 
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6.  Assumptions  about  integration  of  syntnetic  and  operational  training 

We  can  oxpect  somewhat  different  transfer  of  training  depending  on 
whether  the  trainee  alternates  days  on  the  synthetic  trainer  and  on  the  oper- 
ational equipment,  or  whether  he  completes  an  entire  phase  of  training  on  the 
trainer  and  then  gets  into  operational  equipment.  It  is  usually  easier  to 
train  for  transfer  when  the  two  situations  are  side-by-side,  and  this  fact 
may  alter  the  demands  placed  on  training  equipment  hardware, 

7 . Integration  of  assumptions  and  stipulations 

Experience  shows  that  it  is  very  difficult  to  get  responsible  individuals 
to  agree  or  become  committed  to  any  of  the  foregoing  stipulations  about  a pro- 
posed trainer.  Nevertheless,  without  a stipulation  as  to  how  a tool  is  going 
to  be  used  and  for  what  purpose,  the  design  of  a tool  or  instrument  is  bound 
to  come  out  of  a crystal  ball.  First  to  design  some  device  and  then  to 
attempt  to  invent  ways  in  which  to  put  it  to  practical  use  invites  all-around 
inefficiency. 

The  designer  should  therefore  get  comments  on  the  foregoing  points  estab- 
lished explicitly.  Without  them  he  is  a cousin  of  the  engineer  who  was  askod 
to  "design  a device  which  would  train  protoprincia  to  fluticate."  That  is, 
without  clear-cut  definitions  and  opsrational  statements  of  objectives,  the 
designer  suffers  an  unreasonable  handicap  in  the  invention  of  training  devices, 

8.  Abstract  hardware  requirements  from  the  relevant  tasks 

This  step  will  consist  of  no  more  than  an  enumeration  of  equipment  items 
such  as  displays  and  controls  which  appear  in  the  description  of  those  tasks 
selected  for  training.  Strictly  speaking,  programs  are  not  hardware  items, 
but  hardware  provisions  will  need  to  be  made  to  generate  themj  thus  they 
should  be  enumerated  also.  Programs  will  include  samples  of  overall  time- 
sharing of  tasks  included  in  the  training  curriculum. 

Accessories  will  include  seats,  instrument  panels  and  other  hardware 
which  will  support  the  trainee,  the  instructor  and  their  operations. 

Scoring  requirements  — each  control  is  at  least  potentially  subject  to 
trainee  error.  In  addition,  there  may  be  other  forms  of  inadequate  trainee 
response  which  will  be  revealed  in  task  analysis,  especially  if  information 
about  characteristic  trainee  and  operator  error  and  malpractice  is  available. 

Qy  setting  down  scoring  requirements  at  this  time,  no  statement  is  necessary 
as  to  whether  scoring  is  to  be  made  by  special  equipment,  or  by  direct  obser- 
vation of  the  instructor. 

Notes  The  abstracting  of  hardware  requirements  is  applicable  to  any 

trainer,  from  training  aid,  or  demonstrator,  to  a simulator.  Since 

a "simulator"  is  a copy  and  not  an  "abstract"  of  the  operational  device, 

obviously  the  physical  configuration  of  the  work-space  will  be  copied. 
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*Iho  list  of  controls  and  displays  will,  howovor,  be  helpful  in  the 
next  step  of  specifying  the  functional  characteristics  to  be  de- 
signed into  them, 

9.  Integrate  tank  analysis  data,  training  stipulations  and  hardware 
requirements  into  specific  recommendations  for  the  training  device.  ~ 

a.  The  trainee's  station 

This  step  now  goes  from  a statement  of  requirements  to  the  particu- 
lars of  what  will  be  presented  to  the  trainee  in  the  form  of  training- space 
configurations j control-display  intersections;  knowledge  of  results  presenta- 
tions and  scoring;  and  specific  programs  and  conditions  of  practice.  The 
development  of  the  trainee's  station  will  generate  the  principal  requirements 
of  the  instructor's  station.  These  requirements  consist  mainly  of  monitor- 
ing the  inputs  to,  and  outputs  from,  the  trainee's  station,  and  the  control 
of  inputs  to  the  trainee's  station.  (Refer  to  the  schematic  diagram  of  the 
Instructor's  Station,  Chapter  VII,  100.) 

I 

A convenient  format  for  setting  down  each  item  of  equipment  recommenda- 
tion will  include  the  following  categories: 

* 

A.  Utle  of  Equipment  Item 

B.  Fhysical  Properties  Recommended 

C.  Functional  Properties  Recommended,  Including 
Statement  of  Tolerance  Value 

1 

j 

D.  Use  Recommendation 

«t 

E.  Rationale  for  the  Recommendation 

In  jobs  which  call  for  rather  straightforward  sequences  of  operations 
(such  as  bomb-handling  and  loading)  some  short  cuts  may  be  suggested.  A 
group  of  tasks  or  actions  which  are  performed  at  about  the  same  time  in  some 
related  operation  (such  as  bomb-loading  into  aircraft)  can  be  grouped  into 
what  may  be  called  a job  segment.  Specify  the  operational  equipment  required 
for  these  tasks  or  the  job  segment,  Iben  in  parallel  columns  specify  the 
requirements  of  training  equipment  which  will  provide  essential  practice  on 
the  critical  discriminations,  decisions  and  human  motor  responses  involved 
in  the  tasks.  Indicate  also  how  knowledge  of  results  or  indications  of 
response  adequacy  will  be  presented  to  the  trainee  (as  through  the  inspection 
and  comments  of  the  instructor,  or  instrumentally) . 
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b.  The  instructor  'a  station 

The  requirements  of  the  instructor's  station  will  bo  generated 
from  the  requirements  of  the  trainee's  station.  Hardware  for  meeting  the 
requirements  of  the  instructor's  station  may  be  dovised.  The  organization 
of  the  hardware  into  the  instructor's  work  space  is  a problem  of  human 
engineering  of  operational  equipment  for  which  the  reader  is  referred  to 
other  sources.  1 

10.  Checking  the  adequacy  of  the  preliminary  design  from  a human  engineering 
standpoint 

Many  steps  have  intervened  between  the  identification  of  the  man-machine 
system  requirements  and  their  quality  standards  and  tolerances,  and  the  plans 
for  the  training  device.  When  these  plans  have  been  prepared,  it  may  be 
helpful  to  refer  again  to  the  requirjments  imposed  on  the  man-machine  team 
in  terms  of  program  inputs  and  work  outputs.  Take  into  account  the  variables 
in  the  quality  or  performance  standards.  Match  each  of  these  variables  with 
actual  practice  afforded  the  trainee  under  the  range  of  programs  provided  by 
the  training  device.  Check  the  adequacy  with  which  the  trainee  is  given 
knowledge  of  results.  This  knowledge  of  results  should  direct  him  efficiently 
towards  learning  to  meet  the  quality  demands  of  the  system. 

Refer  also  to  the  restrictions  in  degree  and  kind  of  task  stipulated 
for  training  by  the  device. 


1 Baker,  C.  A.,  and  Grethar,  W.  F.  Visual  presentation  of  information. 

WADC  Tfi  51j-l60.  Wright  Air  Development  Center,  Air  Research  and  Develop- 
ment  Command,  Wright  Patterson  Air  Force  Base,  Ohio.  I9!?li, 

Chapanis,  A.,  Garneri,  W.  R.  and  Morgan..  C.  T.  Applied  experimental 
psychology.  New  York:  Wiley  and  Sons,  191$. 

Tufts  College  handbook  of  human  engineering  data.  (2nd  Ed.)  SDC  Tech.  Rpt. 
No.  SDC  199-1-2,  SDC,  ONR,  Port  Washington,  N.  I.  193>1  • 

Woodson,  W.  E.  Human  engineering  guide  for  equipment  designers.  Berkeley, 
Calif:  Dniv.  of  Calif,  Press,  195h. 

Note:  The  Woodson  reference  has  an  excellently  arranged  bibliography 
of  up-to-date  reference  on  representative  problems. 
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When  all  these  matters  are  taken  into  account,  suggestions  for  modifying 
the  tentative  plans  for  the  training  device  may  occur.  Improvements  should 
aim  not  only  to  bring  the  trainee  more  effectively  to  the  operational  skill 
but  at  cutting  the  cost  of  the  training  and  the  training  device. 

11.  Preparing  detailed  instructions  for  the  use  of  the  training  device. 

It  is  reasonable  that  the  considerations  which  go  into  the  design  of  the 
training  device  should  be  reflected  in  the  way  it  is  used  in  training.  Many 
examples  can  be  shown  of  the  manufacturer's  product  being  misused  in  purposes 
not  intended  for  it.  Other  examples  can  be  cited  in  which  the  product's 
capabilities  were  not  exploited  because  they  were  not  known  by  the  user  of  the 
product. 

The  designers  and  manufacturers,  aided  if  possible  by  the  training  per- 
sonnel who  framed  the  need  of  the  training  device,  should  prepare  a detailed 
instructional  syllabus.  Ibis  syllabus  may  be  developed  around  the  task,  job 
segments  and  programs  for  which  training  is  intended  on  the  device.  How  to 
use  the  device  in  accomplishing  the  training  objectives  should  be  incorporated 
in  the  manual  in  the  form  of  detailed,  concrete,  "how-to-do-it"  instructions. 

Obviously  these  instructions  supplied  to  the  user  of  the  device  will  not  rule 
out  changes  and  improvements  in  use  derived  from  experience  with  the  equipment. 

They  will,  however,  tond  to  insure  that  the  intent  of  the  designer  will  be 
realized. 

The  following  reports  cover  in  detail  the  steps  summarized  in  this  chapter: 

Miller,  Robert  B.  A Method  for  Man-Machine  Task  Analysis. 

WADC  TR  53 -13?,  Wright  Air  Development  Center,  Air  Research 
and  Development  Command,  USAF,  Wright-Paaerson  Air  Force 
Base,  Ohio,  1953. 

Miller,  Robert  B.  Human  Engineering  Design  Schedule  for 
Training  Equipment.  WADC  IR  53-138,  Wright  Air  Development 
Center,  Air  Research  and  Development  Command,  USAF,  Wright- 
Patterson  Air  Force  Base,  Ohio,  1953. 

Swain,  Alan  D.  Guide  for  the  Design  and  Evaluation  of  the 
Instructor's  Station  in  Training  Equipment.  WADC  TR  $h-56k, 

Wright  Air  Development  Center,  Air  Research  ard  Development 
Command,  USAF,  Wright-Patterson  Air  Force  Base,  Ohio,  1953. 

Miller,  Robert  B.  A Method  for  Determining  Human  Engineering 
Design  Requirements  For  Training  Equipment.  WADC  TR  53-1 35 j 
Wright  Air  Development  Center,  Air  Research  and  Development 
Command,  USAF,  Wright-Patterson  Air  Force  Base,  Ohio,  1953. 

Miller,  Robert  B.  Handbook  on  Training  and  Training  Equipment 
Design.  WADC  IR  53-136,  Wright  Air  Development  Center,  Air 
Research  and  Development  Command,  USAF,  Wri^it- Patter  son  Air 
Force  Base,  Ohio,  1953. 

I 
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These  references  can  be  obtained  through  t>  Pyachology  Branch,  Aero- 
Kadical  Laboratory,  Wright  Air  Development  Conter,  Wxd^it-Pattorson  Air 
Force  Base,  Ohio. 

Concluding  Comment 

It  should  be  re-eraphasized  that  these  chapters  have  been  devoted  to  the 
problem  of  designing  training  devices  with  minimal  characteristics  necessary 
to  give  the  trainee  the  knowledges  and  skills  required  for  successful  per- 
formance on  the  job— or  at  least  some  portion  of  those  skills  and  knowledges. 

The  success  of  a training  device  should  be  measured  by  its  efficienoy  in  train- 
ing the  skills  intended,  its  transfer  value  to  operations  and  its  economy. 

The  latter  has  significance  not  only  in  dollars  but  in  developmental  effort, arid 
in  the  larger  quantity  of  training  items  so  urgently  needed  for  practice  with- 
out which  no  skill  is  aoquired  and  perfected.  Furtheraore,  engineering 
simplicity  may  decrease  down  time  and  maintenance,  aad  increase  overall 
reliability  of  the  device,  a virtue  of  critical  significance  to  training  schedules. 
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For  a fuller  treatment  of  praotically  any  topic  discussed  on  previous 
pages  the  reader  is  referred  to  the  parent  volumes  from  which  the  present 
report  is  drawn.  Each  of  these  volumes  contains  a reference  bibliography. 
"The  Handbook  on  Training  and  Training  Equipment  Design”  contains  both 
general  and  topical  bibliographies  which  should  be  consulted.  Space  forbide 
the  repetition  of  these  bibliographies  here.  Furthermore,  since  much  of  the 
content  herein  is  a condensation  and  synthesis  of  widely  scattered  reference 
material,  it  is  felt  that  a "selected"  bibliography  here  might  be  more  mis- 
leading than  helpful. 
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Appendix  I 

Sample  Task  Analysis  Page 

Job:  Driving  19$h  Oldemobile  Hydroma tic 


Critical 

stimulus 

variables 


TASK  Tracking  toward  a variable  aperture:  Three  rates  problem 
(Passing  a car  with  an  oncoming  car  in  the  passing  lane). 


DISPIAY  Path  ahead  with  oncoming  oar;  competing  car  ahead  and  to 

Problem  the  right.  Assume  competing  car  will  not  accelerate. 

Critical  (l)  The  absolute  rate  of  competing  car  (I): 

lnpu£  (2)  The  absolute  rate  of  oncoming  car  (Y)j  cars  of 

variables  variable  else  and  speed. 

(3)  The  absolute  momentary  distance  between  X and  I. 

Critical  (1)  Absolute  visual  size  (visual  angle)  of  approaching 
stimulus  car  at  given  instants  down  to  and  including  unsafe 

variables  passing  distances  (distance  estimation). 

(2)  Textural  detail  of  oncoming  car  at  given  instants  of 
approach  (uncertain  cue  value  for  distance  estimation). 

(3)  Position  and  size  of  oncoming  car  relative  to  aerial 
perspective  of  roadway  at  given  instants  of  approach. 

(li)  Rate  of  change  of  variables  (l),  (2)  and  (3)  taking 
into  account  combined  approach  motions  of  oncoming 
car  and  operator's  car. 

(5)  Knowledge  (memory)  of  acceleration  potential  of 
operator's  car  at  that  speed. 

Programs  (l)  Illumination:  Daylight,  night-time,  dusk,  road 
and  " glare,  non-glare. 

noise  (2)  Atmospheric  condition:  Clear,  moderate  rain,  light 


variables  mist. 

(3)  Width  of  roadway:  Sample  of  2li'  two-lane  roadway, 
and  iff*  two-lane  paved  roadway  to  be  found. 

(li)  Size  of  oncoming  car:  Standard  car  size  (Ford,  Buick); 
small  car  slSss  (Nash  Rambler) ; large  car  size  (two- ton 
truck) . 

(ff)  Rat es  of  motion:  Sample  combinations  of  oncoming 
rates  and  competing  car  rates  cf  30  to  80  mph. 

(6)  Subjective  conditions:  Stress  from  anxiety  about 
collision  with  competing  car  (X),  or  running  off 
roadway;  worry  that  competing  car  will  increase 
speed  while  being  passed. 


DECISIONS  Pass  now  or  pass  later. 


GGSTROIS 

Description:  Accelerator,  steering  wheel. 


Activation: 


Steering  wheel  - use  steering  to  wheel  turn  ratio 
characteristic  of  operator's  car.  Introduce  maximum 
play  in  steering  allowable  in  State  inspections. 

(Continued  oat  nest  page) 
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Accelerator!  Rydr&aatio  arrangement  - for  rapid 
access  of  power,  press  all  the  way  to  floor-board 
past  a resistance  detent, 

Actiont  Accelerator  response  cited  drops  transmission  from 

lith  to  3rd  gear  below  6&**?2  Kfh*  More  power  from 
faster  engine -to -wheel  ratio. 


FEEDBACK 

Cues 


Time  delay 

Criterion  of 
response  ade- 
quacy 

Critical 

Values 

Corrective 

action 


Sane  as  display,  but  as  X,  I and  operator  approach, 
the  success  of  tho  solution  or  lack  of  it  becomes 
more  apparent. 

Function  of  acceleration  rate  of  operator's  car, 

Ostting  back  into  ri^it  lane  with  "safety  margin" 
(50'  plus)  depending  on  speed,  (Bettor  criterion 
would  be  in  terms  of  time  between  turning  in  to 
right  lane  and  collision!  $ seconds,) 

Collision  course  perceived  imminent 


Brake  and  return  to  right  lar.ej  ditch  on  loft  side 


CHARACTERISTIC 

(1) 

ERRORS  AND 

MALFUNCTIONS 

(2) 

(3) 

(« 

(5) 

Getting  too  close  behind  X before  accelerating 
and  passing  X, 

Failing  to  take  into  account  slight  grade. 

Failing  to  take  into  account  cars  behind  0 which 
may  prevent  return  behind  X in  emergency. 

Not  returning  to  right  path  as  soon  as  safe  to  do  so. 
Attempting  to  return  to  right  path  too  soon. 
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Accessories,  12l* 

Associations,  cue,  73 
Assumptions,  stipulations.  12l* 
Attention,  dividing,  1*5 
Attitude,  39 

Changing  responses  10,  12f,  ll* 
Coding,  3h,  109#  121 
Communication,  speech,  106f 
Context 

operational,  33 
response,  51 

Continuities  of  trai.nee  response,  8l 

Controls,  105ff 

avoidance  of  looking  at,  1*5 
configurations,  U6 
display  directional  relationships 
37 

display  ranges,  39 

display  raticy  38 

display  relations,  36 

display  time  characteristics,  U9 

forces,  U6 

settings,  108 

Coordination,  response,  2,  16 

Criterion,  89 

progress  towards  final,  75 

Cue  associations,  73 

Cues 

artificial,  ?2 
available  in  operations,  1*9 
critical,  in  complex  backgrounds, 
1*2 

dramatic,  72 
feedback,  67 


Cues  (continued) 

intermittent  presentation,  1*5 
interpretability,  69 
spurious,  73 

supporting  or  context,  1*9 
task-relevant,  1*2 

Degree  of  training,  123 

Delay,  66 

Demonstration,  111 
equipment,  32 
functions  of,  ll5ff 
motor  aspects,  111 
perceptual  aspect,  111 

Demonstrator 

as  a basis  for  generalizing,  117 
as  a working  framework,  117 
of  principles,  115-121 
working  components,  117-121 

Design,  preliminary,  126 

Diagrammatic  photographs,  36 

Differentiation,  response,  2 

Display,  36 

considerations,  102ff 
legibility,  103 
of  trainee* a performance,  102 
repeater,  101* 
type,  103 

(See  also  control  display) 

Dividing  attention,  1*5 

Economy,  20,  23 

Enhancement  of  critical  cues,  1*2 

Error 

constant  vs«  variable,  8l 
directional,  70 
machine.  76,  96 
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Snt)r,  (continued) 

perception  of,  78 
signal,  identifying,  77 


Information,  (continued) 
stored,  12 

transmitting  information  to  trainee,  106-111 


Environmental  context  of  operations, 31 

Equipment,  operational,  29,  33 

Evaluator,  92-95 

selection  of  sample  programs,  92 
selection  of  tasks,  92 
selection  of  variables,  9kf 

Exercises,  39 

Fatigue,  13 

Feedback 

learning,  7f,  8,  63,  66,  67,  69, 
73,  71*,  78 
operational,  714-78 
pointing  to  stimulus  component, 68 
precision  of,  68 
signals,  symbolizing,  67 
specificity,  69 
action,  9,  62,  71,  78 
artificial,  71,  72 
channels,  7 
cues,  67 
determining,  78 
informational,  62,  76 
information,  use  of,  69 -7U 
lag,  37 

Freeze  switch,  108 
Grouping 

by  isolation,  103 
functional,  103 
sequential,  103 

Groups  of  items,  3h 

Hardware  requirements,  125f 

Ideational  processes,  6 

Information 
coded,  107 

perceiving  in  larger  blocks,  1*1 


i Inputs  to  trainee's  station,  109f 

Instructional  personnel,  123 

Instructions  for  use,  127 

Instructor 

controls,  105f 
general  facilities,  112f 
location,  101 
overloaded,  82 

(See  also  instructional  personnel) 

Instructor's  station,  98, -111*,  126 
equipment  considerations,  99f 
layout,  99 

Integration 

of  assumptions  & stipulations,  121* 
of  part-task  outcomes,  32 
temporal,  1*1 
of  training,  12  ii 

Intermediate  stages  . *'  task  learn- 
ing, 1*0-52 

Intermittent  presentation  of  cues,  1*5 

Interspersing  simulated  operational 
target  programs,  5l 

Knowledge  of  results,  8,  61-80 
function  in  training,  6l 
immediate,  3h 

Learning 

control-display  relations,  36 
definition,  1 

early  stages  in  task  learning,  31-1*0 
of  procedures,  31 
stage  of,  31-60,  69 
what  the  equipment  is  supposed 
to  do,  32 

Location  of  objects,  33 
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Man-Machine  teams,  operational,  32 

Measurement,  proficiency,  81-97 

Mock-ups 

functional,  37 
non-functional,  3k 
or  panel  displays,  36 

Monitoring,  88,  103 

Motiviation,  1-lljf 

Motor  activities,  perceiving  signals 
in  advance  of  initiating,  h$ 

Motor 

"'ovem8nts,  coordinating,  h5f 
processes,  6 

Nomenclature,  33 

Operability 

checks,  101? 
of  training  device,  101 

Operational,  29,  32,  37 
equipment,  29,  32,  37 
man-machine  teams,  32 
meaning,  103 

target  course,  programs  of,  1$ 
Operations 

environmental  context  of,  32 
normal,  109 

Order  of  target  course  presentation, 


Orient  to  what  next  to  learn,  62 
Oscillation,  response,  68 
Overloaded  instructor,  82 
Part 

of  performance,  82 
simulators,  29 
task,  32,  70 

Pattern^  1*3 

response,  2 


Perceiving 

information  in  larger  blocks,  I4I 
signals  in  advance  of  initiating 
motor  activities,  14i 

Perceptual 

processes,  6 
sensitivity,  17 
simplicity,  10  2f 

Phase  of  training,  32 


Pictures,  3)4 

Practice  exercises,  37 

Precision,  120 

PrLmary  work  area 

stimuli  outside,  3 5 
stimuli  tfithin,  35 

Proficiency  measurement,  89-97 

Program 

data,  sampling  of,  28 
of  target  couisos,  1$ 
of  exercises,  39 
range  of  target,  50 
simulated  operational  target,  $1 

Progress  towards  final  criterion,  75 

Range  of  target  programs,  50 

Rate  of  stimulus  movement,  2*8 

Recommendations 
Specific,  125 

(See  also  trainer  recommendations) 

Recorded  data  as  a help  in  teaching,  81 

Recording 

backlash,  87 
equipment,  102,  82-86 
equipment,  82 

for  evaluational  purposes,  62* 
procedures,  81-88 
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Recording  (continued) 
range,  86 
reliability,  8? 
sensitivity,  86 
validity,  87 

Reinforcement,  73 

Relations  among  parts  of  performance,  82 

Reliability 
human,  96f 
recording,  87 

Representing  transmission  systems,  120 

Responding  to  rates  of  stimulus  move- 
ment, U8 

Response,  7 

adequacy,  67 
chains,  hi 
changing,  10-lb 
contexts,  5 1 
continuities,  81 
coordination,  2 
differentiation  If 
inadequate  modes,  73 
new,  7 

oscillation,  68 
pattern,  2 
review,  71 
serialization,  2f 
serial  motor,  b7 

Sampling,  27-29,  70 
of  program  data,  28 

Scanning,  35 

Scoring  75-77 

and  reference  baselines,  95 
equipment,  102 
requirements,  12b 

Search,  scanning  methods,  35 

Self 

pacing,  70 

perception  of  error,  68 

Sensitivity,  perceptual,  17 


Serial  motor  responses,  3,  b? 

Simulation 

degree  of  physical,  31-60 
operational  target  programs,  51 
physical,  19-30 
psychological  vs«  engineering, 
19-23 

Simulators,  32 
part,  29 

Speech  communication,  106f 

Stage  of  learning,  31-60,  69 

Stimulus,  11,  6b 

critical,  recalling,  66 
differentiation,  b 
generalization,  b 
import,  lb 
movement,  b8 
patterning,  5f 
symbolizing,  66 
variation,  17 

Trainee  population,  123 

Trainee  station,  109f 

Symbols 

behavior,  12 
ideational  activity,  12 
processes,  6 

Target  courses,  b9 
presentation,  5l 
programs,  5l 

Task 

analysis,  122f 
continuous  feedback,  18 
procedural,  18 
to  be  trained,  123 
total,  70 

Teams,  man-machine,  32 

Temporal  integration,  bl. 

Tolerance  graduation,  75-76 
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Trainers 

number  required,  33 
part-task,  29 
procedural,  29 

Training  process  and  human  learning, 

Chap.  I 

Training,  stages  of  and  learning,  Chap.  IV 

Transfer,  20-23 

and  type  of  task,  18 
of  training,  16 

Transition  training,  58 

Transmission  system,  representing,  120 

Trouble  consoles,  108 

Unique  solution  problems,  50 

Use 

instructions  for,  127 
recommendations  for,  121 

Variation 

between  units,  26 
of  examples,  120 
within  units,  26-27 

Variety  of  target  programs,  5l 

Verbal  instruction  about  error  signals,  69 

Verbalization  of  correct  stimulus -re spouse,  70 

Work  area,  primary,  35 


